Close

Success!

A project log for ISS HDEV image availability

The "High Definition Earth Viewing" experiment onboard the ISS runs a few cameras, but not always. Let's find out when video is available!

christophChristoph 08/17/2018 at 08:400 Comments

I tried writing a python script that would use streamlink to connect to a stream, grab a frame every now and then, and analyse it. Whatever I tried, it would either be horribly slow or the stream would time out after just a few seconds.

What works, though, is using ffmpeg as a player at a low-ish framerate and dump images to disk:

streamlink -O http://ustream.tv/channel/iss-hdev-payload worst | ffmpeg -i - -r $fps -f image2 -update 1 out.jpg

Now I have a more or less recent frame every few seconds (I picked 0.2 fps for my application). I found that out.jpg from that command is usually ahead (think 10 to 20 seconds) of the image shown by the ustream web viewer.

The command shown above runs in an endless loop in the cloud, so I don't have to keep a server running at home. The reason for the endless loop is that sometimes the stream times out or has other problems, and streamlink would then terminate.

Now out.jpg can be analysed. Since it's now just a regular file, I had to check if it has been modified since the last image analysis. I tried pyinotify for this purpose and it seems to work well. Caveat: ffmpeg seems to write it three times per update, and each of these accesses generates a notification. A little deadtime between analyses fixed this. The analysis is a structural similarity measurement between the recently grabbed frame (out.jpg) and a snapshot of the "no stream available" image. Then the result is published to the mosquitto test broker. Here's the quick and dirty code:

import pyinotify
import time

from skimage.measure import compare_ssim
import imutils
import cv2

import paho.mqtt.client as mqtt

threshold = 0.9

class EventHandler(pyinotify.ProcessEvent):
  template_img = cv2.cvtColor(cv2.imread("novideo.jpg"), cv2.COLOR_BGR2GRAY)
  template_width = template_img.shape[1]

  def __init__(self):
    self.start_time = time.time()
    self.last_event = 0
    self.deadtime = 1
    self.updated = False
    self.client = mqtt.Client("analyse_client")
    self.client.connect("test.mosquitto.org", 1883)
    self.client.loop_start()

  def analyse(self):
    self.updated = True
    now = time.time()
    if now > (self.last_event + self.deadtime):
      recent_img = cv2.cvtColor(cv2.imread("out.jpg"), cv2.COLOR_BGR2GRAY)
      recent_width = recent_img.shape[1]
      f = float(self.template_width)/float(recent_width)
      recent_img = cv2.resize(recent_img, (0,0), fx=f, fy=f)
      (score, diff) = compare_ssim(recent_img, self.template_img, full=True)
      t = now - self.start_time
      available = score < threshold
      print("modified @t = {:.3f} s; score = {:.3f}; available = {}".format(t, score, available))
      self.client.publish("iss-hdev-availability/available-bool", int(available))
    self.last_event = now

  def process_IN_CLOSE_WRITE(self, event):
    self.analyse()

def main():
  wm = pyinotify.WatchManager()  # Watch Manager
  handler = EventHandler()
  notifier = pyinotify.ThreadedNotifier(wm, handler)
  notifier.start()
  wdd = wm.add_watch('out.jpg', pyinotify.IN_CLOSE_WRITE)

  last_event = handler.last_event
  while True:
    try:
      time.sleep(30)
      if handler.updated:
        handler.updated = False
      else:
        print("no updates, exiting")
        break
    except KeyboardInterrupt:
      print("keyboard interrupt, exiting")
      break

  #wm.rm_watch(wdd.values())
  notifier.stop()

if __name__ == "__main__":
  main()

This script is also running in an endless loop in the cloud since notifications won't come anymore after the grabbing command was restarted. I don't know why, but a loop does the job.

So if you just want to know if an actual live image from the ISS HDEV experiment is available (without having to look at the actual video output), the topic "iss-hdev-availability/available-bool" on test.mosquitto.org can be subscribed to by anyone. Here's a screenshot from my phone:

Of course it also works with a command line tool:

mosquitto_sub -h test.mosquitto.org -t iss-hdev-availability/available-bool

Discussions