Vidgear – A complete, cross-platform video processing Python framework

VidGear is a powerful python Video Processing library built with multiple APIseach with a unique set of trailblazing features. These APIs provides an easy-to-use, highly extensible, multi-threaded & asyncio wrapper around many underlying state-of-the-art libraries such as OpenCV ➶ , FFmpeg ➶ , ZeroMQ ➶ , picamera ➶ , starlette ➶ , pafy ➶ and python-mss ➶

The following functional block diagram clearly depicts the generalized functioning of VidGear library:

Table of Contents

• Gears: What are these?
• New-Release SneekPeak: v0.1.7
• • Available Installation Options
• Testing, Formatting & Linting
• Contributions & Support

TL;DR

What is vidgear?

"VidGear is an ultrafast➶ , compact, flexible and easy-to-adapt complete Video Processing Python Library."

What does it do?

"VidGear can read, write, process, send & receive video frames from/to various devices in real-time."

What is its purpose?

"Built with simplicity in mind, VidGear lets programmers and software developers to easily integrate and perform complex Video Processing tasks in their existing or new applications, without going through various underlying library's documentation and using just a few lines of code . Beneficial for both, if you're new to programming with Python language or already a pro at it."

For more information, see Frequently Asked Questions ➶ .

Gears

VidGear is built with multiple APIs (a.k.a Gears) , each with some unique function/mechanism.

Each of these APIs is exclusively designed to handle/control different device-specific video streams, network streams, and media encoders. These APIs provide an easy-to-use, highly extensible, multi-threaded and asyncio wrapper around state-of-the-art libraries under the hood to exploit their internal parameters and methods flexibly while providing robust error-handling and unparalleled performance.

These Gears can be classified as follows:

A. VideoCapture Gears:

• Targets various IP-USB-Cameras/Network-Streams/YouTube-Video-URL.
• Targets various Raspberry Pi Camera Modules.
• Enables ultra-fast Screen Casting.
• Common API with Video Stabilizer wrapper.

B. VideoWriter Gear:

• Handles easy Lossless Video Encoding and Compression.

C. Network Gears:

• Targets flexible video-frames and data transfer between interconnecting systems over the network.

• Asynchronous I/O Network Gears:

  • ASGI Video Server that transfers live video frames to any web browser on the network.
  • Fast, Memory-Efficient Asyncio video-frame messaging framework.

CamGear

CamGear can grab ultra-fast frames from diverse range of devices/streams, which includes almost any IP/USB Cameras, multimedia video file format ( upto 4k tested ), various network stream protocols such as http(s), rtp, rstp, rtmp, mms, etc. , plus support for live Gstreamer's stream pipeline and YouTube video/live-streams URLs.

CamGear provides a flexible, high-level multi-threaded wrapper around OpenCV's VideoCapture class with access almost all of its available parameters and also employs pafy python APIs for live YouTube streaming . Furthermore, CamGear implements exclusively on Threaded Queue mode for ultra-fast, error-free and synchronized frame handling.

Following functional block diagram depicts CamGear API's generalized workflow:

CamGear API Guide:

>>> Usage Guide

VideoGear

VideoGear API provides a special internal wrapper around VidGear's exclusive Video Stabilizer class.

Furthermore, VideoGear API can provide internal access to bothandAPIs separated by a special flag. Thereby, this API holds the exclusive power for any incoming VideoStream from any source, whether it is live or not, to access and stabilize it directly with minimum latency and memory requirements.

Below is a snapshot of a VideoGear Stabilizer in action ( See its detailed usage here ):

_{Original Video Courtesy @SIGGRAPH2013}

Code to generate above result:

# import required libraries
from vidgear.gears import VideoGear
import numpy as np
import cv2

# open any valid video stream with stabilization enabled(`stabilize = True`)
stream_stab = VideoGear(source = "test.mp4", stabilize = True).start()

# open same stream without stabilization for comparison
stream_org = VideoGear(source = "test.mp4").start()

# loop over
while True:

    # read stabilized frames
    frame_stab = stream_stab.read()

    # check for stabilized frame if Nonetype
    if frame_stab is None:
        break

    # read un-stabilized frame
    frame_org = stream_org.read()

    # concatenate both frames
    output_frame = np.concatenate((frame_org, frame_stab), axis=1)

    # put text over concatenated frame
    cv2.putText(
        output_frame, "Before", (10, output_frame.shape[0] - 10), cv2.FONT_HERSHEY_SIMPLEX,
        0.6, (0, 255, 0), 2,
    )
    cv2.putText(
        output_frame, "After", (output_frame.shape[1] // 2 + 10, output_frame.shape[0] - 10),
        cv2.FONT_HERSHEY_SIMPLEX,
        0.6, (0, 255, 0), 2,
    )

    # Show output window
    cv2.imshow("Stabilized Frame", output_frame)

    # check for 'q' key if pressed
    key = cv2.waitKey(1) & 0xFF
    if key == ord("q"):
        break

# close output window
cv2.destroyAllWindows()

# safely close both video streams
stream_org.stop()
stream_stab.stop()

VideoGear API Guide:

>>> Usage Guide

PiGear

PiGear is similar to CamGear but made to support various Raspberry Pi Camera Modules (such as OmniVision OV5647 Camera Module and Sony IMX219 Camera Module ) .

PiGear provides a flexible multi-threaded wrapper around complete picamera python library to interface with these modules correctly, and also grants the ability to exploit its various parameters like brightness, saturation, sensor_mode, etc. effortlessly.

Best of all, PiGear API provides excellent error-handling with features like a Threaded Internal Timer - that keeps active track of any frozen-threads/hardware-failures robustly, and exit safely if it does occur. So now if someone accidentally pulls Camera module cable out, when you're running PiGear API in your script, instead of going into possible kernel panic or frozen threads, this API will exit safely to save resources.

Code to open picamera stream with variable parameters in PiGear API:

# import required libraries
from vidgear.gears import PiGear
import cv2

# add various Picamera tweak parameters to dictionary
options = {"hflip": True, "exposure_mode": "auto", "iso": 800, "exposure_compensation": 15, "awb_mode": "horizon", "sensor_mode": 0}

# open pi video stream with defined parameters
stream = PiGear(resolution = (640, 480), framerate = 60, logging = True, **options).start() 

# loop over
while True:

    # read frames from stream
    frame = stream.read()

    # check for frame if Nonetype
    if frame is None:
        break


    # {do something with the frame here}


    # Show output window
    cv2.imshow("Output Frame", frame)

    # check for 'q' key if pressed
    key = cv2.waitKey(1) & 0xFF
    if key == ord("q"):
        break

# close output window
cv2.destroyAllWindows()

# safely close video stream
stream.stop()

PiGear API Guide:

>>> Usage Guide

ScreenGear

ScreenGear API act as Screen Recorder, that can grab frames from your monitor in real-time either by define an area on the computer screen or fullscreen at the expense of insignificant latency. It also provide seamless support for capturing frames from multiple monitors.

ScreenGear provides a high-level multi-threaded wrapper around python-mss python library API and also supports a easy and flexible direct internal parameter manipulation.

Below is a snapshot of a ScreenGear API in action:

Code to generate the above results:

# import required libraries
from vidgear.gears import ScreenGear
import cv2

# open video stream with default parameters
stream = ScreenGear().start()

# loop over
while True:

    # read frames from stream
    frame = stream.read()

    # check for frame if Nonetype
    if frame is None:
        break


    # {do something with the frame here}


    # Show output window
    cv2.imshow("Output Frame", frame)

    # check for 'q' key if pressed
    key = cv2.waitKey(1) & 0xFF
    if key == ord("q"):
        break

# close output window
cv2.destroyAllWindows()

# safely close video stream
stream.stop()

ScreenGear API Guide:

>>> Usage Guide

WriteGear

WriteGear handles various powerful Writer Tools that provide us the freedom to do almost anything imagine with multimedia files.

WriteGear API provide a complete, flexible & robust wrapper around FFmpeg , a leading multimedia framework. With WriteGear, we can process real-time video frames into a lossless compressed format with any suitable specification in just few easy lines of codes . These specifications include setting any video/audio property such as bitrate, codec, framerate, resolution, subtitles, etc. easily as well complex tasks such as multiplexing video with audio in real-time(see this example wiki ). Best of all, WriteGear grants the freedom to play with any FFmpeg parameter with its exclusive custom Command function(see this example wiki ), while handling all errors robustly.

In addition to this, WriteGear also provides flexible access to OpenCV's VideoWriter API which provides some basic tools for video frames encoding but without compression.

WriteGear primarily operates in the following two modes:

• Compression Mode:In this mode, WriteGear utilizes FFmpeg inbuilt encoders to encode lossless multimedia files. It provides us the ability to exploit almost any available parameters available within FFmpeg, with so much ease and flexibility and while doing that it robustly handles all errors/warnings quietly. You can find more about this mode here .

• Non-Compression Mode:In this mode, WriteGear utilizes basic OpenCV's inbuilt VideoWriter API . Similar to compression mode, WriteGear also supports all parameters manipulation available within this API. But this mode lacks the ability to manipulate encoding parameters and other important features like video compression, audio encoding, etc. You can learn about this mode here .

Following functional block diagram depicts WriteGear API's generalized workflow:

WriteGear API Guide:

>>> Usage Guide

NetGear

NetGear is exclusively designed to transfer video frames synchronously and asynchronously between interconnecting systems over the network in real-time.

NetGear implements a high-level wrapper around PyZmQ python library that contains python bindings for ZeroMQ - a high-performance asynchronous distributed messaging library that aim to be used in distributed or concurrent applications. It provides a message queue, but unlike message-oriented middleware, a ZeroMQ system can run without a dedicated message broker.

NetGear provides seamless support for Bi-directional data transmission between receiver(client) and sender(server) through bi-directional synchronous messaging patterns such as zmq.PAIR (ZMQ Pair Pattern) & zmq.REQ/zmq.REP (ZMQ Request/Reply Pattern) .

NetGear also supports real-time Frame Compression capabilities for optimizing performance while sending the frames directly over the network, by encoding the frame before sending it and decoding it on the client's end automatically in real-time.

For security, NetGear implements easy access to ZeroMQ's powerful, smart & secure Security Layers, that enables Strong encryption on data , and unbreakable authentication between the Server and the Client with the help of custom certificates/keys and brings easy, standardized privacy and authentication for distributed systems over the network.

Best of all, NetGear can robustly handle Multiple Server Systems at once, thereby providing access to seamless Live Streaming of the multiple device in a network at the same time.

NetGear as of now seamlessly supports three ZeroMQ messaging patterns:

• zmq.PAIR (ZMQ Pair Pattern)
• zmq.REQ/zmq.REP (ZMQ Request/Reply Pattern)
• zmq.PUB/zmq.SUB (ZMQ Publish/Subscribe Pattern)

Whereas supported protocol are: tcp and ipc .

Following functional block diagram depicts generalized workflow of NetGear API in its Multi-Servers Mode:

NetGear API Guide:

>>> Usage Guide

WebGear

WebGear is a powerful ASGI Video Streamer API, that transfers live video frames to any web browser on the network in real-time.

WebGear API provides a flexible abstract asyncio wrapper around Starlette ASGI library and easy access to its various components independently. Thereby implementing the ability to flexibly interact with the Starlette's ecosystem of shared middle-ware and mountable applications & seamless access to its various Response classes, Routing tables, Static Files, Template engine(with Jinja2), etc.

WebGear can acts as robust Live Video Streaming Server that can stream live video frames to any web browser on a network in real-time. It also auto-generates necessary data files for its default template and provides us the freedom to easily alter its performance parameters and routing tables according to our applications while handling errors robustly.

In addition to this, WebGear provides a special internal wrapper aroundAPI, which itself provides internal access to bothandAPIs thereby granting it exclusive power for streaming frames incoming from any device/source. Also on the plus side, since WebGear has access to all functions ofAPI, therefore it can stabilize video frames even while streaming live.

Below is a snapshot of a WebGear Video Server in action on the Mozilla Firefox browser:

_{WebGear Video Server at http://0.0.0.0:8000/ address.}

Code to generate the above result:

# import required libraries
import uvicorn
from vidgear.gears.asyncio import WebGear

#various performance tweaks
options = {"frame_size_reduction": 40, "frame_jpeg_quality": 80, "frame_jpeg_optimize": True, "frame_jpeg_progressive": False}

#initialize WebGear app  
web = WebGear(source = "foo.mp4", logging = True, **options)

#run this app on Uvicorn server at address http://0.0.0.0:8000/
uvicorn.run(web(), host='0.0.0.0', port=8000)

#close app safely
web.shutdown()

WebGear API Guide:

>>> Usage Guide

NetGear_Async

NetGear_Async can performance boost upto 1.2~2x times as compared toat about 1/3 of memory consumption but only at the expense of limited modes and features.

NetGear_Async is an asyncio videoframe messaging framework built on zmq.asyncio and powered by high-performance asyncio event loop called uvloop to achieve unmatchable high-speed and lag-free video streaming over the network with minimal resource constraint. Basically, this API is able to transfer thousands of frames in just a few seconds without causing any significant load on your system.

NetGear_Async provides complete server-client handling and options to use variable protocols/patterns similar tobut doesn't support any NetGear Exclusive modes yet. Furthermore, NetGear_Async allows us to define our own custom Server Source to manipulate frames easily before sending them across the network(see this wiki example).

NetGear_Async as of now supportsall four ZeroMQ messaging patterns:

• zmq.PAIR (ZMQ Pair Pattern)
• zmq.REQ/zmq.REP (ZMQ Request/Reply Pattern)
• zmq.PUB/zmq.SUB (ZMQ Publish/Subscribe Pattern)
• zmq.PUSH/zmq.PULL (ZMQ Push/Pull Pattern)

Whereas supported protocol are: tcp and ipc .

Code for NetGear_Async Server-Client API:

NetGear_Async API Guide:

>>> Usage Guide

New Release SneekPeak : VidGear 0.1.7

:warning: Dropped support for Python 3.5 and below legacies.

• WebGear API:

  • Added a robust Live Video Server API that can transfer live video frames to any web browser on the network in real-time.
  • Implemented a flexible asyncio wrapper around starlette ASGI Application Server.
  • Added seamless access to various starlette's Response classes, Routing tables, Static Files, Templating engine(with Jinja2), etc.
  • Added a special internal access to VideoGear API and all its parameters.
  • Implemented a new Auto-Generation Workflow to generate/download & thereby validate WebGear API data files from its GitHub server automatically.
  • Added on-the-go dictionary parameter in WebGear to tweak performance, Route Tables and other internal properties easily.
  • Added new simple & elegant default Bootstrap Cover Template for WebGear Server.
  • Added __main__.py to directly run WebGear Server through the terminal.

• NetGear_Async API

  • Designed NetGear_Async asynchronous network API built upon ZeroMQ's asyncio API.
  • Implemented support for state-of-the-art asyncio event loop uvloop at its backend.
  • Achieved Unmatchable high-speed and lag-free video streaming over the network with minimal resource constraint.
  • Added exclusive internal wrapper around VideoGear API for this API.
  • Implemented complete server-client handling and options to use variable protocols/patterns for this API.
  • Implemented support for all four ZeroMQ messaging patterns: i.e zmq.PAIR , zmq.REQ/zmq.REP , zmq.PUB/zmq.SUB , and zmq.PUSH/zmq.PULL .
  • Implemented initial support for tcp and ipc protocols.

• Asynchronous Enhancements

  • Added asyncio package to vidgear for handling asynchronous network APIs.
  • Various performance enhancements for these Asyncio APIs for achieving concurrency within a single thread.

• Added new highly-precise Threaded FPS class for accurate VidGear benchmarking with time.perf_counter python module and many more...

Documentation

The complete documentation for all VidGear APIs and functions can be found in the link below:

• Wiki Documentation - English

Installation

Prerequisites:

Before installing VidGear, you must verify that the following dependencies are met:

• Supported Systems:

  VidGear is tested and supported on the following systems withand pip already installed:

  • Any Linux distro released in 2016 or later
  • Windows 7 or later
  • macOS 10.12.6 (Sierra) or later

• Supported Python legacies:

  • Python 3.6+ are only supported legacies for installing Vidgear v0.1.7 and above.

• Pip Dependencies:

  Wheninstalling VidGear with pip, you need to install following dependencies manually:

  • OpenCV:Must Require OpenCV(3.0+) python binaries installed for its core functions. For installation, you can either follow these complete online tutorials for Windows , Linux and Raspberry Pi , or just install it directly via pip:

    $ pip install -U opencv-python       # or install `opencv-contrib-python` similarly

  • FFmpeg:Must Require FFmpeg for its video compression and encoding compatibilities inAPI.

    :star2: Follow this FFmpeg wiki page for its installation. :star2:

  • Picamera:Must Required if you're using Raspberry Pi Camera Modules( such as OmniVision OV5647 Camera Module ) with itsAPI. You can easily install it via pip:

    $ pip install picamera

    :bulb: Also, make sure to enable Raspberry Pi hardware-specific settings prior to using this library.

  • Uvloop:Only Required if you're using itsAPI on UNIX machines for maximum performance. You can easily install it via pip:

    :warning: Uvloop is NOT yet supported on Windows Systems .

    $ pip install uvloop

Available Installation Options:

Option 1: PyPI Install(recommended)

Best option for quickly getting VidGear installed.

:warning: Seebefore installing!

# Installing stable release
  $ pip install vidgear

  # Installing stable release with Asyncio support
  $ pip install vidgear[asyncio]

Option 2: Release Download

Best option if you want a compressed archive .

VidGear is available for download as wheel( .whl ) package in our release section, and can be installed with pip as follows:

:warning: Seebefore installing!

# directly installs the wheel
  $ pip install vidgear-{downloaded version}-py3-none-any.whl

Option 3: Build from source

Best option for trying latest patches( maybe experimental ), Pull Requests , or contributing to development.

You can easily clone the repository's latest testing branch, and thereby install it as follows:

$ git clone https://github.com/abhiTronix/vidgear.git
  $ cd vidgear
  $ git checkout testing
  $ pip install .[asyncio]           # installs all required dependencies including asyncio

Testing, Formatting & Linting

Requirements:

Testing VidGear require some additional dependencies & dataset that can be downloaded manually as follows:

• Install additional python libraries:

  $ pip install --upgrade six
    $ pip install --upgrade flake8
    $ pip install --upgrade black
    $ pip install --upgrade pytest
    $ pip install --upgrade pytest-asyncio

• Download Test Dataset:

  To perform tests, additional test dataset is required, which can be downloaded (to your temp dir) by running bash script as follows:

  $ chmod +x scripts/bash/prepare_dataset.sh
    $ .scripts/bash/prepare_dataset.sh               #for Windows, use `sh scripts/bash/prepare_dataset.sh`

Running Tests:

• Pytest:Then, tests can be run with pytest ( in VidGear's root folder ) as follows:

  $ pytest -sv                                   #-sv for verbose output.

Formatting & Linting:

For formatting and linting, following tools are used:

• Flake8:You must run flake8 linting for checking the code base against the coding style (PEP8), programming errors and other cyclomatic complexity:

  $ flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics

• Black:Vidgear follows black formatting to make code review faster by producing the smallest diffs possible. You must run it with sensible defaults as follows:

  $ black {source_file_or_directory}

Contributions & Support

Contributions are welcome! Please see our Contribution Guidelines for more details.

Support

Love using VidGear? Consider supporting the project to fund new features and improvements

Contributors

Community Channel

We're on Gitter :star2: ! Please join us.

Citing

Here is a Bibtex entry you can use to cite this project in a publication:

@misc{vidgear,
    author = {Abhishek Thakur},
    title = {vidgear},
    howpublished = {\url{https://github.com/abhiTronix/vidgear}},
    year = {2019}
  }

Copyright

Copyright © abhiTronix 2019

This library is released under the Apache 2.0 License .