Feature Request: adding pixel_dif in the mjpeg stream

Description

OK Here is my last angle on this request

Embed in the Jpeg image Motion information as comments using the jpeg_write_marker function.

I have not had enough time to dissect the motion structures as they are not well commented so I threw in a bunch of attributes from the motion context structure just for testing.

1.

I made a copy of the put_jpeg_yuv420p function and gave it the signature as below where picInfo is a new parameter and is the data to embed in the jpeg

void put_jpeg_yuv420p_with_info (FILE *fp, unsigned char *image, int width, int height, int quality, char *picInfo)

2.

I added the jpeg_write_marker to the new function put_jpeg_yuv420p_with_info at the point as shown below

   jpeg_stdio_dest (&cinfo, fp);   
   jpeg_start_compress (&cinfo, TRUE);
   jpeg_write_marker(&cinfo, JPEG_COM, picInfo, strlen(picInfo));

3.

In function put_picture_fd I added the following (only test so not robust but to show the concept);

char msgBuffer[250];
sprintf(msgBuffer, "Diffs=%i\nNoise=%i\nThreshold=%i\nSnapShot=%i\nPevent=%i\nType=%i\n", cnt->diffs,cnt->noise,cnt->threshold,cnt->snapshot, cnt->prev_event, cnt->imgs.type);

and changed put_picture_fd to call

put_jpeg_yuv420p_with_info(picture, image, cnt->imgs.width, cnt->imgs.height, quality, msgBuffer);

Now

• The mjpeg stream remains intact as per protocol
• This is a valid Jpeg
• Nothing is broken or convoluted.

The strings above were for testing to see what values change. Really the most important attribute is Diffs.

What do you think ?

Modified Code from picture.c

void put_picture_fd (struct context *cnt, FILE *picture, char *image, int quality)
{
   
       char msgBuffer[250];
   sprintf(msgBuffer, "Diffs=%i\nNoise=%i\nThreshold=%i\nSnapShot=%i\nPevent=%i\nType=%i\n", cnt->diffs,cnt->noise,cnt->threshold,cnt->snapshot, cnt->prev_event, cnt->imgs.type);

   
   if (cnt->conf.ppm) {
      put_ppm_bgr24(picture, image, cnt->imgs.width, cnt->imgs.height);
   } else {
      switch (cnt->imgs.type) {
         case VIDEO_PALETTE_YUV420P:
            //put_jpeg_yuv420p(picture, image, cnt->imgs.width, cnt->imgs.height, quality);
            put_jpeg_yuv420p_with_info(picture, image, cnt->imgs.width, cnt->imgs.height, quality, msgBuffer);
            break;
         case VIDEO_PALETTE_GREY:
            put_jpeg_grey(picture, image, cnt->imgs.width, cnt->imgs.height, quality);
            break;
      }
   }
}





void put_jpeg_yuv420p_with_info (FILE *fp, unsigned char *image, int width, int height, int quality, char *picInfo)
{
   int i,j;

   JSAMPROW y[16],cb[16],cr[16]; // y[2][5] = color sample of row 2 and pixel column 5; (one plane)
   JSAMPARRAY data[3]; // t[0][2][5] = color sample 0 of row 2 and column 5

   struct jpeg_compress_struct cinfo;
   struct jpeg_error_mgr jerr;
   
   data[0] = y;
   data[1] = cb;
   data[2] = cr;
   
   
   cinfo.err = jpeg_std_error(&jerr);  // errors get written to stderr 
   
   jpeg_create_compress (&cinfo);
   cinfo.image_width = width;
   cinfo.image_height = height;
   cinfo.input_components = 3;
   jpeg_set_defaults (&cinfo);

   jpeg_set_colorspace(&cinfo, JCS_YCbCr);

   cinfo.raw_data_in = TRUE; // supply downsampled data
   cinfo.comp_info[0].h_samp_factor = 2;
   cinfo.comp_info[0].v_samp_factor = 2;
   cinfo.comp_info[1].h_samp_factor = 1;
   cinfo.comp_info[1].v_samp_factor = 1;
   cinfo.comp_info[2].h_samp_factor = 1;
   cinfo.comp_info[2].v_samp_factor = 1;

   jpeg_set_quality (&cinfo, quality, TRUE);
   cinfo.dct_method = JDCT_FASTEST;

   jpeg_stdio_dest (&cinfo, fp);       // data written to file
   jpeg_start_compress (&cinfo, TRUE);
   jpeg_write_marker(&cinfo, JPEG_COM, picInfo, strlen(picInfo));


   for (j=0;j<height;j+=16) {
      for (i=0;i<16;i++) {
         y[i] = image + width*(i+j);
         if (i%2 == 0) {
            cb[i/2] = image + width*height + width/2*((i+j)/2);
            cr[i/2] = image + width*height + width*height/4 + width/2*((i+j)/2);
         }
      }
      jpeg_write_raw_data (&cinfo, data, 16);
   }

   jpeg_finish_compress (&cinfo);
   jpeg_destroy_compress (&cinfo);
}

-- RobertH - 30 May 2005

Follow up

Hmm. Interesting idea.

Maybe a good implementation would be a config option "jpeg_marker_text" which allows all the same rules as on_xxxx like time stamp, diffs, noise, coordinates etc. And let this be a general option for both normal jpegs and the mjpeg stream. If the string is empty - no marker is written.

It may open up for all sorts of usage. The code overhead is small and the user can select to use it or not.

The main problem is that this is a feature at level 4 on Kenneths NerdoMeter. But since I am proposing it myself ...

Implementation is simple since the function for writing the string based on conversion specifiers is already there and the info is in the global "cnt" structure.

I am interested in reactions from both Robert and the regular developers.

Robert, you have another feedback which I agree with. The global cnt variable is not very well documented. I have made a topic MotionCntStructureOverview which should list all the variables or members in a table. It will probably take a while before this table is complete but it is a start.

-- KennethLavrsen - 30 May 2005

Comments

Kenneth, I really like the idea of a config option such as you propose as it stays in line with the current rules and offers a lot of flexibility. The only point since this information will always be read by software and not a person we would also be required to embed the format string used to construct the data.

So we might end up with [formatString] [some delimiter] [actualData] as our payload.

-- RobertH - 30 May 2005

---

The method used with the conversion specifiers allow any combination of text and %-something codes. So it would be fully flexible.

E.g. jpeg_marker_text "diffs = %D, x = %K, y = %L"

or jpeg_marker_text "<D,K,L><%D><%K><%L>"

I'd rather keep things flexible. And anyone who develops viewer software can then specify the string to be used for the jpeg_marker_text option.

-- KennethLavrsen - 31 May 2005

---