GST Motion

Introduction

GST Motion is an ongoing project aiming at moving Motion video/image features into the GStreamer framework. GSt Motion will be a new application that uses the motion source code.

Plan

• Develop GStreamer plugins GSt Motion needs:
  • GSturl for downloading from netcams [done]
    • Decode mjpeg [planning]
  • Video4linux & Video4linux2 already [part of GSt-plugins]
  • Round robin [?]
• Compile motion as a GStreamer plugin
  • Compile motion as a library [planning: MotionLibrary]
  • Decide how to pass streams [to discuss]
  • Add some GST functionality to motion [if needed]
• Change motion deeply to become more GST oriented
  • Dynamic pads [far in future]
  • Spiders [far in future]

More details

This topic was created from a Motion Mailing List posting 28 May 2005 from VincenzoDiMassa. I have edited the wording to fit as a discussion intro.

If you read MotionRearchitectureDiscussion and Kenneth comments on that wikipage you will remember that we agreed upon:

1. dividing motion into 3 parts.
2. motion needs to be as little and fast as possible.
3. motion should be newby friendly (and possibly even hacker friendly).
4. motion is one executable with only basic dependencies.

So, the first part of motion has to grab imges form *camera.

JeroenVreeken suggested using GStreamer as a framework to use for implementing the new input system.

GStreamer is a framework for handling streams of data.

The basic concept of GStreamer is the Element: one element can be a source, a filter or a sink of data. Elements are connected using Pads (pads can be source pads or sink pads). Pads ate typized using mime/tipes and Caps (Capabilities for a video stream could be for example bpp, width, height...). Known and common data types are already handled by the GStramer media library.

Elements are generally provided as plugins.

Coding new elements is very easy. Coding a motion Element would be a great addition for motion itself. Suppose a new video format/codec/driver needs to be used (for example quicktime or xvid or v4l3 or...): to support this new codec motion has to modify its code its config options and its buid system:

1. add code to implement the functionality
2. add new options to enable the the new format/codec/driver compile
3. add new options in the motion.conf to let users configure the format/codec/driver specific features
4. update the docs

If motion were using a framework like GStreamer the only thing needed would be GStreamer itself (with support for the new format/codec/driver).

No recompile of motion (so no code, build system and options change) would be needed.

This is because motion will just need be able to interpret a video/x-raw-rgb or video/x-raw-int stream. All the steps needed to bring the data to the raw formats are done outside of motion inside GStreamer itself.

The input plugin to can be specified in the motion conf file: this means you can add 100+ input methods but you will just need to adjust ONE setting in the motion conf file.

For example:

gst-launch urlsrc  "url=http://localhost:8000/image.jpg" ! jpegdec ! motion ! xvidencode ! avimux ! filesink "location=/var/motion/file.avi"

The above command line can be created using a nice GTK GUI (GStEditor). GStEditor allows users to draw pipelines visually (connectiing blocks). See http://gstreamer.freedesktop.org/modules/gst-editor.html

GStreamer is so little footprint and fast that Nokia (http://gnomedesktop.org/node/2265) is actually using it on embedded devices.

I am attaching a figure to show what I am talking about.

http://gstreamer.freedesktop.org/data/doc/gstreamer/head/manual/html/chapter-intro.html has a nice introduction to GStreamer.

Downloads

• gst-url-0.1.0.tar.bz2: GSturl plugin for GStreamer

Discussion

AngelCarpintero had these concerns

1. It uses libcurl , as motion did for old netcam code . We stop using libcurl due the low performance we got with it.
2. Your plugin doesn't support mjpeg , are you going to implement it ?
3. Maybe you can use gsttcp instead of curl in order to have a better control and error handlers.

VincenzoDiMassa answered

1. About curl: I would like to test alternatives to curl as long as they are more robust and/or hacker friendly.
2. About mjpeg streams I am already planning to implement code to using them too. It isn't already in because the mjpeg streaming camera my university owns is not available at the moment.

---

I am still not convinced that this whole thing is the way I want to see Motion go.

I have been on the Motion project for 3 years writing all the documentation and writing code for various sections. So I guess I know most of Motion from the inside now.

But what you are trying to do is still a bit of a mystery to me and if I do not understand it - no newbie use will ever. It may be simpler than I think but in that case you still have not communicated what it is that is going on here.

I fear that what this all leads to is that the user has to install and configure and understand different new applications (Gstreamer for example) and figure out how to get them to play together.

I fear we are at level 9 on the KennethsNerdoMeter. At least this is what I feel at the moment. If I do not understand this - noone will - except a few programmers that happen to know GStreamer.

gst-launch urlsrc  "url=http://localhost:8000/image.jpg" ! jpegdec ! motion ! xvidencode ! avimux ! filesink "location=/var/motion/file.avi"

Syntax like this belongs inside a program - not in users interfaces. Users should never see horror like this.

Some important notes in line with MotionMissionStatement.

• Motion is ONE executable with basic dependencies. And with basic dependencies I mean standard shared libraries that are available for all distributions and that users install without any knowledge of what they do and without any configuration. You should not have to know anything about Gstreamer to use Motion! Just install a few RPMs or deb packages or whatever and you are done.
• Will this Gstreamer idea still work with 16 cameras in parallel without 20 GB of RAM? How stable is the code? Can it run 24/7/365.
• What about all the feature interactions we have today. There are 1000s of hours of coding getting all this to play together.
• A newbie that has used Linux for 5 days. Does he stand a chance? I still think Motion needs to be simplified and the config files made more intuitive. I keep on changing things a little by little while improving the documentation also. There are 1000s of hours of work behind the documentation alone. Who will document the new features?
• When I said I would like to see inputs and outputs with a plugin support then it was to make writing interfacing code simpler. Today the V4L/Netcam code and the different outputs Motion can do - are not made with a clean well defined API so it is difficult to plugin additional functions. And it bugs me that an RPM of Motion requires ffmpeg, MySQL, PostgresSQL RPMs to be installed even if you do not plan to use for example PostgresSQL. A simple plugin system where the bits and pieces are loaded only if included in the config file would be great. A bit like Apache does. But as a user the only thing I have to do in Apache is uncomment ONE line that loads the plugin and define the config options that belongs to it. That is simple.
• Basic plugins that gives the functionality that we know today must be part of Motion. Most should be built-in. There is no reason why getting picture frames from V4L/V4L2 and saving jpeg pictures should be moved outside Motion. That only complicates things.
• New plugins should be something you install in a directory, point to in ONE line in motion.conf and you can use it by adding config options specific to the plugin. And we should be dammed careful with the syntax of existing options. Again consult the KennethsNerdoMeter. Level 2 is my target. This is a bit like plugins are done for this TWiki. You download a Perl module and a TWiki plugin topic. Copy them to the TWiki directories and it works. Settings for the plugin are on the plugin topic. Simple. Like any plugin feature should be.

I may sound a bit negative. But I see my mission as to protect the stability and user friendliness of this project. I do not rule out continuing with your ideas. But as long as I do not understand the practical implementation - noone will. I started reading the Gstreamer docs. And I still do not feel I really understand anything.

-- KennethLavrsen - 04 Jun 2005

---

Kenneth, the idea of using GStreamer comes from the needs of creating a modular architecture to add options/features to motion while keeping the basic motion executable as simple as it deserves to be.

• Why GStreamer? Because it is little and fast. Again: "GStreamer is so little footprint and fast that Nokia (http://gnomedesktop.org/node/2265) is actually using it on embedded devices". So it should robust as well.
• I think that to keep things simple we use standards. GStreamer is becoming a standard framework to develop multimedia/streaming/video applications. Users of the Gnome desktop or users of the Nokia 770 will know about GStreamer. KDE is considering adopting GStreamer. This is becoming the standard way to handle multimedia development.
• Why should one spend time to study motion custom config file options, setup, install, build when he could do the some thing using the knowledge he already has?
• The great work you are doing on motion and the directions you are following are something all OOS project should have. I really like how you manage (and how you care of) the health of this nice project. I'm not willing to transform into a horror!
• There is no need to move anything outside motion! V4L/V4L2 can live both inside and outside motion. One day will came V4L3 comes out... will motion users need to enable it inside motion? Wouldn't it be better to just attach motion to the new bleeding edge V4L3 driver (using a nice GUI or something we consider more user friendly)?
• Twiki plugins are plugins that work with one app only. GStreamer plugins are very different! Twiki uses all the installed plugins. Motion will have to enable just ONE input plugin per device (Netcam/V4L/V4L2). Haw can you just copy somewhere the plugin for a Netcam that streams MPEG4 and hope it needs no configuration? What is the ip/port the camera is streaming from? GStreamer has a nice solution for this kind of problems allowing plugin specific config options. This does not make the motion.conf file more complicated.
• GST Motion aims at gradually moving thing around. It will be (if you agree) just a GStreamer plugin that is compiled from motion source code. It can be made an optional and well insulated makefile target.

<joke mode>
What is KennethsNerdoMeter level of KennethsNerdoMeter itself? Is a newbie able to understand it? I think complex things require skilled users. Just don't force users who don't need complex things to learn them. Things should be kept as simple as possible and when it is not possible the must be optional.
</joke mode>

-- VincenzoDiMassa - 10 Jun 2005

---

There is one important element here.

• Will the Gstreamer be a plugin to Motion?
• Or will Motion be a plugin to Gstreamer?

If it is the latter - I do not see the point. Why would we do that? I cannot imagine any code being more efficient than the current Motion code that fetches the video frames straight from the device.

I see a fragmented thing comming out of it.

You will have to explain how the "application" would be built because I do not get that yet.

KennethsNerdoMeter is my message to nerds (I am a nerd myself) that 99% of Motion users are not nerds. And that Motion is currently too complicated to use and I want it made simpler - not more difficult.

"I think complex things require skilled users". I completely disagree. I do not want an application where normal users are only able to use a fraction of the features. I want all the complex features available to all and easy to use.

It is always possible to make complex things simple. It takes time and effort and it requires that you try and think like a person who is not a programmer and never will become a programmer.

I will not let Motion become a nightmare to setup like for example sendmail.

Another concern: GStreamer manual says QUOTE " In order to understand this manual, you will need to have a basic understanding of the C language. Since GStreamer uses GLib 2.0, the reader is assumed to understand the basics of the GObject object model. It is recommended to have skimmed through the introduction of the GObject tutorial before reading this. You may also want to have a look at Eric Harlow's book Developing Linux Applications with GTK+ and GDK." UNQUOTE.

I do not know Glib. I do not understand or even know the basics of the GObject model. And I do not know GTK+ and GDK either. Today Motion is written in standard C and except for the ffmpeg code which is poorly documented I have gained a good overview of what is going on inside Motion. I have the feeling that GSTMotion is going to be a full time job to be familiar with and I only have about 1 hour per day to spend on this project.

The GStreamer manual seems to be a big mouthfull to learn on my own.

-- KennethLavrsen - 11 Jun 2005

---

Sorry for my later reply. I still have in mind a GSTMotion app. I spent my summer coding for google Summer of Code, now I have courses, so GSTMotion is waiting.

In my mind motion will never know about GStreamer at all, so don't worry about the internals. The motion application can be sligthly modified to:

• allow a gstreamer plugin to push data into it
• allow motion itself to push data into a gstreamer plugin

This way motion can be used as a gstreamer plugin and motion can use gstreamer plugins. I already did this to MotionHuMoR: I just added to MotionHuMoR ~10 lines of code. The main problem is that motion can't be used as a library, but that has nothing to do with GStreamer. If some parts of motion are compiled into a lib than both the current motion app and any other app will be able to use that lib (that lib can be a static lib if you don't like dlls related problems). So, Gstreamer hadling will be 98% out of motion source code. The 2% will be a couple of function calls that copy data to and from pads.

I'm still here (a bit busy for the while) and want to add this functionality to motion as a goal of my Ph.D., do you still have concerns?

-- VincenzoDiMassa - 27 Sep 2005

---

You are not the only one that has suggested Gstreamer related ideas and I have decided that instead of me being a road block preventing crazy ideas - I want Motion architecture changed so that Motion gets a plugin interface. We will start with input device plugins since this is where the need is greatest. Followed by output plugin for making movies.

See PluginArchitectureDiscussion where we have started the discussion. The plugin architecture is still in the discussion phase and inputs are welcome.

The end goal is that Motion is shipped with a set of standard default plugins with the same functions as today plus a few new.

The plugin API will be well documented and kept as stable as possible and feasible.

And people can create and post plugins freely and upload them to the Motion Twiki for people to download.

And a Gstreamer set of plugins - maybe several implementations of them - can then be realized without asking anyone for permission and without needing to convince anyone.

When I look at your suggestion to turn Motion into a library I wonder how you would make Motion features such as

• multiple cameras
• round robin - the exclusive access to same input device
• http control - independent thread
• webcam feature - how to maintain the state of connected cameras
• Mix of USB cams, composite cams (bttv) and network cams

Motion is much more than just feeding a single video stream in and getting video out from a single camera.

-- KennethLavrsen - 28 Sep 2005

---

Kennet, I did not explain myself clearly: I don't want motion to turn into a library. Motion needs to be an app. My proposal is to develop the features of motion into libs that the motion app calls. It is just a matter of defining an API for some of the motion internals. For example you could want a library that has an API like this:

• a function to register a camera id (a string like "Camera1")
• a function to process one frame from a given camera like process(img,"Camera1) (this function does everything that motion can do)

This simple piece of API is something that could be exported/usefull to apps other than motion. Another kind of API would enable motion be used without motion detection: suppose you already have a "trigger" to detect intrusion and you just want to write an app to process the event.

So, what I mean by "making motion a library" is to make motion define an API that an external app writer can use to "steal" motion functionalities. That API will be beneficial for motion too because:

• it will help plugin writers documenting how motion works
• it will make motion components more insulated and roboust
• it will let other projects better cooperate with motion code exposing motion to more eyes
• it will let me write a GSTmotion GStreamer plugin

-- VincenzoDiMassa - 03 Oct 2005

---

* Will the Gstreamer be a plugin to Motion? * Or will Motion be a plugin to Gstreamer?

As I see, both. Motion (detection algorithm) would be abstracted to a new Gstreamer element, while Motion (application) would use Gstreamer elements like v4l and jpeg decode/encode.

I See Kenneth is criticizing GStreamer for being to complex for developers and users. Fair enough, but reinventing your own plugin architecture is a worse idea. Especially since MotionRearchitectureDiscussion suggests "Pipes&filters architecture", which looks a lot like GStreamer arch. NiH is bad, you will have only few eyes viewing at motion plugin architectucture, while GStreamer has many eyes fixing and improving.

For developers, it is better to persuade GStreamer developers to make more comprehensible docs, that to implement a similar but motion-specific system.

For users, motion should hide GStreamer under hoods - it should be in no way more visible to users than say, using libjpeg in current motion.

I'd also like to criticize your "plugins should be installable by copying to a directory". That is a security maintenance nightmare. Vast majority of cracked Linux boxes is due to php applications "just copied to a directory".

Such installations never get security updates in time, unlike software installed via package management (apt-get, emerge, yum etc). And when software is installed via package management, installation is no longer a enduser, but packager issue.

-- RikuVoipio - 12 Dec 2005

---

Any updates on this? - I would love to use motion inside gstreamer - would let me pre-capture outside of motion and write the original stream instead of transcoding with motion

Been playing with decodebin and uridecodebin and they seem to make all the GSTurl and other stuff redundant, no?

Would just be nice to have something like vrlsrc (or urldecodebin) ! motion option1=value option2=value ! fakesink and have that work

Internally the gstreamer app or script can query motion to get x/y coordinates, change options live, etc.

-- RomanGaufman - 01 Jul 2011

---

-- KennethLavrsen - 29 May 2005