Feature Request: Mechanism to query for updated motion statistics

Description

Either through the "HTTP Based Control" or via a read-only named-pipe on the filesystem which is running Motion, it would be very useful to have Motion output one line per input or thread. This output looking similar to the following.

<Camera thread number or identifier> <total pixels changed ala a router statistic> <pixels changed since last polled> <pixels changed on the current image> <number of motion events triggered to date> <number of motion events triggered since last polled> <current threshold value>\n

The "<total pixels changed ala a router statistic>" would be a forever increasing total of all Motion-pixels-changed for the input or thread (as in the "text_changes on" option). When this counter hits the maximum value for its data type (unsigned long?) it simply rolls over. This should act exactly like ifconfig on Linux acts (as in RX bytes:<value>). This is because rrdtools and other utilities are able to handle a forever increasing value.

The "<pixels changed since last polled>" is not as valuable as the other one. If this one is possible to implement (which I'm not sure of,) it would clear the value after the it is polled and start totalling it up as input or a thread is viewed for motion.

The "<pixels changed on the current image>" would simply be whatever value the "text_changes on" option in Motion would display on the current output image.

The "<number of motion events triggered to date>" would reflect a constantly increasing value which gets incremented by one for each time Motion creates media based on motion being detected.

The "<number of motion events triggered since last polled>" would show the number of motion-detection events that occurred since this was last polled. It is cleared after this value is polled. It is incremented by one each time motion is detected.

The "<current threshold value>" would be useful in tuning. For instance, if you have a single camera, you can graph the # of pixels changed over a day, and also graph the threshold during that time--you can see at a glance that perhaps you are missing some motion that would have been captured had your threshold been low enough. Or perhaps you are detecting too much motion and need to turn on smart_mask'ing, etc.

The point of all this is to take the numbers given and graph them. You could have a graph that displayed the # of pixel changes for, let's say, 8 webcams during a 4 hour period. At a glance you can see what the busy times were and expect to find media generated by Motion during those times. It could also be useful in tuning.

I'd expect, possibly, a parameter in the .conf that would allow you to specify the named-pipe for each thread. Or perhaps a single pipe for all threads. Also, this might be served up in the proc filesystem, but this might be too unix-specific? All of these parameters would be unsigned longs. Also, it might be usefull to also have a parameter which would be thread/cam specific to format the output in the same manner you can format the image paths being written ("jpeg_filename".)

This would be quite a feature, and I'm sure everyone can understand its usefulness.

-- PeterS - 21 Oct 2005

Follow up

Comments

Named pipes, proc file system, new config options, etc. All a bit high on KennethsNerdoMeter.

If we introduce a statistics feature it will be in the http control interface and without the need of config options.

And whatever is introduced should not put any extra load on the Motion execution. Counting every changed pixel would be an activity on every picture frame captured times every camera. If you look at the extra CPU power added during setup mode you will know what I mean. But maybe we can improve the setup mode. And maybe we can use some of the data we already have in a statistics http page.

Let us discuss and brain storm this a bit more. I hear the message. You need more data to work with for tuning and analysis/presentation of captured images.

-- KennethLavrsen - 21 Oct 2005

---

Well, I can envision a few different implementations that would either be high on your NerdoMeter or low on it.

Lowest NerdoMeter value would be employ the use of rrdtool within Motion itself to provide the graphs for each cam and/or all cams so that you can see this via the "HTTP Based Control", for tuning, etc, without any configuring on the user's side. Another implementation would be just to give the numbers (so that a person can manually graph, set alerts, etc externally) either via an external tool binary (sorta like running 'ps' to get stats from the kernel,) cat'ing a named pipe, via HTTP and the "HTTP Based Control", or via "Setup Mode" (I don't like that last idea.)

The CPU time required would be practically NIL as this feature would only be using data that has already been collected from the code that implements the "text_changes on" parameter (or the "Draw the number of changed pixels on the images" parameter.) So, the data is already there--all we have to do is tabulate it over time and cough it up to the user(s). Uber-low overhead.

At the minimum I'd like to see config options like "statistics_pipe off" and "statistics_pipe_filename". It defaults to off, so no worries for inexperienced users. The "statistics_pipe_filename" would go in the "Target Directories and filenames For Images And Films" section of motion.conf . The pipe-file would get generated in the directory specified by the "target_dir" option. To get data from it you simply do "cat <path/filename>". Simple.

What I need, really, is statistics for pixel change, localized to time, so that long-term maintenance of Motion would be easier (using pipes or a helper bin and not good to have in HTML form.) Not necessarily for tuning (this would be useful, but is very short-term, although would lend itself well to being HTML-ized.)

As time permits, I'll try and implement something myself to provide at least a useful example.

-- PeterS - 21 Oct 2005

---

You seem to have missed the MotionMissionStatement.

The view that narrow scoped difficult to understand features do no harm because the newbie can just ignore them is wrong. This is why so many programs are so impossible for anyone to setup. How do you know which features are important? How do you know just to leave certain options as default values? No new options just so you can make some graphs. And the naming: statistics_pipe?? Newbies will ask me "What have you been smoking?" for adding an option like that.

Let us look at what you try to do instead of jumping right into implementation?

Do you want to write out the changed pixels for each frame captured? That is going to be a lot of disk writing. Or only when Motion is detected? You can actually put this data in a MySQL DB with the feature set available today.

The problem with your feature is that it is narrow (few people will ever use it), adds complexity which may be OK if a lot of people benefit from it and it will add load no matter what. You talked tuning. And you talk statistics. And data that resets when you read it. A very specific implementation that you have in mind.

Now try and see the feature in a broader scope.

Look at the discussion GenericTrackingPatch. Look at todays on_xxxx commands. Look at conversion specifiers. Try and think more generic. Instead of implementing narrow features, it is much better to consider making more broad features. Let us brainstorm a bit.

• on_xxxx features could be made more versatile. Maybe they should be able to do more without having to spawn a shell process. Maybe connect to a socket and send data or get a URL. Maybe write information to a file.
• add a few more on_xxxx events (which will add load but only when used)
• Create a new socket which when connected to it streams lines of text to the connected client. The information is given in a option which will hit the max allowed score of 4 on KennethsNerdoMeter. But with a broad scope. Same socket can be used for tracking. For prediction feature. My idea is that the format is 100% freely defined using conversion specifiers. You can use it for your statistics. I can play with tracking. Some other guy may want something else.
• Same think as above - reusing the http control ports. A certain URL used gives you the stream (server push once connected)
• The http control method: Why not two user defined modes?

Going generic allows much more freedom in creating new ideas and without having to add 2-3 new options for each idea.

Continue the brainstorm.

-- KennethLavrsen - 21 Oct 2005

---

Patch topic StatisticsDataPatch has been created for PeterS to post some concept code that may later become a new feature in Motion.

-- KennethLavrsen - 25 Oct 2005

---

I did not realize that you had created something prior that is very similar to this, but yet embedded in the output media itself.

http://www.lavrsen.dk/twiki/bin/view/Motion/FeatureRequest2005x05x30x143842

Perhaps the two goals can be combined. I'm not sure. It may be that the same informational text designated to be included in the output media can be displayed for each thread on the Statistics page. They are just so similar.

-- PeterS - 29 Oct 2005

---

Request as such is accepted but the implementation in the current patch is a little too crude in its user interface. I will at some point work with Angel getting it better integrated into the http control interface. It has not been given much priority because of other activities. And Peter's personal needs are temporarily resolved with his own patch which still should work for 3.2.4.

-- KennethLavrsen - 18 Dec 2005

---