BUG: Motion quits if any netcams falter at startup.

The motion netcam code is pretty good at recovering from errors, but it appears to be designed to quit all threads if there are problems retrieving images from any of the cameras at startup.

Thus, In an auto-start environment (e.g. recovering from a power-failure) motion won't start up until the faulty cameras are removed from the config file.

Test case

1. Start motion with multiple network cameras (ensure that the first camera is unreachable); In my test case I use 5 network cameras.

2. The output (stdout) received is similar to:

Tue Sep 27 20:14:16 PDT 2005 - Starting start_motion.sh
[0] Processing thread 0 - config file motion.conf
[0] Processing config file /usr/config/camera1.properties
[0] Processing config file /usr/config/camera2.properties
[0] Processing config file /usr/config/camera3.properties
[0] Processing config file /usr/config/camera4.properties
[0] Processing config file /usr/config/camera5.properties
[1] Thread is from /usr/config/camera1.properties
[1] Thread started
[2] Thread is from /usr/config/camera2.properties
[2] Thread started
[3] Thread is from /usr/config/camera3.properties
[3] Thread started
[4] Thread is from /usr/config/camera4.properties
[4] Thread started
[5] Thread is from /usr/config/camera5.properties
[5] Thread started
[0] motion-httpd/3.2.3 running, accepting connections
[0] motion-httpd: waiting for data on port TCP 8080
[4] Unrecognized image header (404)
.
.
.
[4] Unrecognized image header (404)
[2] connect returned error
[2] Failed to open camera - check your config
[2] Failed to read first camera header - giving up
[2] Capture error calling vid_start
[2] Thread finishing...

Then motion quits (/var/log/messages show almost identical output to this as well).

From the output it appears that motion had no problems with cameras 1,3, and 5.

Environment

Motion version:	3.2.3
ffmpeg version:	ffmpeg-cvs-20050813-certified-with-motion-3.2.2
Shared libraries:	ffmpeg
Server OS:	2.4.21-27.0.4.EL #1 Sat Apr 23 06:30:55 EEST 2005 i686 athlon i386 GNU/Linux

-- BruceDurham - 28 Sep 2005

Follow up

Reading through the code (in motion.c, routine motion_init) the current program logic says that if any camera fails during startup the program exits (i.e. it's not a particular netcam feature, but rather a more general one). Failure is defined as an error (negative) return code from the camera startup routine.

We want to be able to do at least a little "validation" of the configuration details, in order to assure the basic startup proceeds in the manner the user intended. In your particular test case, one of the failing netcams was producing unreadable data, and the other was not responding to a connect request. Both of these symptoms can occur if the user has set an invalid configuration, so I'm not convinced we want to ignore them, or even just let the startup proceed. On the other hand, I understand your point about recovery, and also understand that although a camera might be currently not present (or even currently mis-behaving) it may be acceptable because the user knows it will be "coming back soon". Perhaps we could consider having a runtime switch to say "ignore camera errors on startup"?

-- BillBrack - 30 Sep 2005

---

I agree 100% that some fixing is needed here. And we need to figure out the best way of doing it. Let me start with some proposals. The main issue is when Motion runs in daemon mode. When you start Motion and a camera cannot be connected to there can be two reasons.

• The camera device or URL is not there. It could in principle be connected later.
  • In this case we would rather let Motion start running and accept the camera when it becomes available. This is relatively easy to do for Netcams but require some major redesign in the initialization code for the camera threads with a slow retry scheme.
• The camera device or URL is wrong and has to be corrected. In this case we would like to swear at the user and tell him to fix things.

Some proposals that has been suggested.

A switch (command line or config file?) that tells Motion to ignore errors.

Yes we can ignore errors. But if we only do that in the current code I am not sure the code can recover. Some parts of the initialization will have to be rewritten so it retries. We cannot just reinitialize everything at each frame because this will cost cycle time. But we can keep on reinitializing until the device is there.

If we somehow can report the error on the console - before we go in daemon mode - maybe avoid this switch so that Motion retry scheme is not depending on people remembering the switch.

Recover failed camera both at startup and when disconnecting while running.

And if we implement a retry scheme at startup this why not let the same scheme recover when a USB camera is unplugged and then replugged. We started the design of this scheme. It works for netcams and the plan was to make it work for other cams as well.

Watchdog

The never implemented watchdog feature is also a design proposal.

When a thread dies - or never startup - instead of killing Motion or preventing it from starting up - a watchdog can keep on trying trying to reinitiate the thread.

I still think the watchdog feature is a great idea but I would not let this be the first recovery method for a lost camera. My doing it in the motion thread by handling errors properly we can show the grey image in the webcam stream and write a smart message that helps people correcting a mistake.

But I have seen problems with Motion starting up after a boot also because MySQL was not ready. And then Motion dies because of that. There are more errors that could make Motion fall over. Motion should be more robust and not like an old motor cycle engine that you cannot easily start after the winter is over.

http control should show error messages

One of the newbie problems is that they never know to look in the syslog for errors. So if we let Motion start with a wrong URL or device name and just let the daemon retry and retry and log messages in the syslog they will never know what happened.

But maybe we can let the http control interface show the latest error messages. Maybe we can make this feature so you enter a URL that gives you the current status of all the camera threads. It can then say that the camera is not available and why.

Grey image if camera of any type not available.

We should use the gray image to indicate that camera is not available.

And write on the image an intelligent error message.

Let Motion die during startup in some cases?

Should Motion die in some cases?

The fact that Folkert made a small program that waits for a camera device before proceeding to startup Motion tells me that this is a real problem and that we should have this taken care of in Motion. Folkert's program should not be necessary.

Quick fix only for Netcams

I propose we help Bruce by making a quick fix that works for Netcams only and continue with the more general issues later. The initialization of netcams is nill so it is very easy to simple report the "wrong URL or camera not available" and let Motion continue with grey image. But I would like to see the message on the console even when Motion is set for daemon mode. But as a quick fix we can do without. Dying in silence or running with grey image in silence. None of these behavours are very newbie friendly anyway.

Feel free to add more ideas or modify the ones above.

-- KennethLavrsen - 30 Sep 2005

Fix record

For the network cameras I have worked on a fix.

Please try the new MotionRelease3x2x4snap4

-- KennethLavrsen - 30 Oct 2005

Released in 3.2.4

-- KennethLavrsen - 18 Dec 2005