July 2023 Update: Most free RTSP trial streams have stopped working. We have modified the default stream to a metered service provided by RTSP.stream company. If the stream does not work, you can open a trial account and test the Colab Notebook; alternatively, you can directly stream your own RTSP if it is internet-facing or run the notebook locally, using Jupiter Lab or similar if your stream is local.
In our previous publication, we leveraged javascript into Colab to obtain a feed from a webcam local to your computer. Unfortunately, our knowledge of javascript is deficient; we really do not work with it, so when trying to obtain the same results for a streaming IP camera delivering a Real-Time Streaming Protocol (RSTP) video feed, we found ourselves placing the worst hack possible into action. First of all, at the point of writing this publication, we do not know how to modify appropriately and effectively the javascript-generated webcam stream; on the other hand, we do know how to alter that stream to overlay detection information, so instead of overlaying detection information into the webcam stream we are going to overlay RSTP feeds from external cameras.
We are not repeating the code from our previous publication. This new overlaying function will add external RSTP feeds to our webcam input, completely masking it under an alpha channel with a value of 1. Again, we modify the image's resolution in case we hit video streams at high resolutions.
The overlay function that covers our webcam feed entirely and replaces it with the streaming video from the external source is this:
def overlay_rtsp(image, output_image):
stream = 'rtsp://zephyr.rtsp.stream/pattern?streamKey=c7c43bbdc298356ee022ea6289706b66'
cap = cv2.VideoCapture(stream)
ret, frame = cap.read()
size = (600, 800)
frame = cv2.resize(frame, size)
output_image[:, :, 0:3] = frame
output_image[:, :, 3] = 1
# Add our logo if present:
try:
logo_file = '/content/ostirion_logo.jpg'
img = cv2.imread(logo_file)
new_size = (100, 100)
img = cv2.resize(img, new_size, interpolation = cv2.INTER_AREA)
lim = -new_size[0]-1
output_image[lim:-1, lim:-1, 0:3] = img
output_image[lim:-1, lim:-1, 3] = 1
except:
pass
return output_imageWe can also modify the drawing array function to accept different video stream sources, overlaying RTSP feeds by default:
def get_drawing_array(image_array, video_width=800, video_height=600,
overlay_function=overlay_rtsp):
drawing_array = np.zeros([video_width, video_height, 4], dtype=np.uint8)
drawing_array = overlay_function(image_array, drawing_array)
drawing_array[:, :, 3] = (drawing_array.max(axis=2) > 0 ).astype(int)*255
return drawing_arrayThe main streaming function can be launched, and the test RTSP should appear:
start_input()
label_html = 'Capturing IP Camera Stream.'
img_data = ''
while True:
js_reply = take_photo(label_html, img_data)
if not js_reply:
break
image = js_reply_to_image(js_reply)
drawing_array = get_drawing_array(image, overlay_function=overlay_rtsp)
drawing_bytes = drawing_array_to_bytes(drawing_array)
img_data = drawing_bytesThis test RTSP feed is taken from this external source and may stop working; if you have your own internet-facing camera, you can change the URL in the overlay RTSP function to use your own. The rendering process is slow; getting a new frame takes a few seconds, even if the source feed is at 30 or 60 FPS. However, it is sufficient for demonstration. You can open the test feed using VLC or similar video software to check the stream rate without being processed by the Colab Notebook.
This should be the notebook output video:
Do not hesitate to contact us if you require quantitative model development, deployment, verification, or validation. We will gladly help you with your machine learning or artificial intelligence challenges when applied to asset management, automation, or intelligence gathering from satellite, drone, or fixed-point imagery.
The notebook for this demonstration is in this link.