Play video frame by frame performance issues

I want to play a video (mostly .mov with Motion JPEG) in frame by frame mode with changing framerate. I have a function who gives me a framenumber and then I have to jump there. It will be mostly in one direction but can skip a few frames from time to time; also the velocity is not constant. So I have a timer asking every 40ms about a new framenumber and setting the new position. My first approach now is with DirectShow.Net (Interop.QuartzTypeLib). Therefore I render and open the video and set it to pause to draw the picture in the graph

FilgraphManagerClass media = new FilgraphManagerClass(); media.RenderFile(FileName); media.pause();

Now I will just set a new position

media.CurrentPosition = framenumber * media.AvgTimePerFrame;

Since the video is in pause mode it will then draw every requested new position (frame). Works perfectly fine but really slow... the video keeps stuttering and lagging and its not the video source; there are enough frames recorded to play a fluent video. With some performance tests I found out that the LAV-Codec is the bottleneck here. This is not included directly in my project since its a DirectShow-Player it will be cast through my codec pack I installed on my PC.

Ideas:

  • Using the LAV-Codec by myself directly in C#. I searched but everyone is using DirectShow it seems, building their own filters and not using existing ones directly in the project.
  • Instead of seeking or setting the time, can I get single frames just by the framenumber and draw them simply?
  • Is there a complete other way to archive what I want to do?

Background:

This project has to be a train simulator. We recorded real time videos of trains driving from inside the cockpit and know which frame is what position. Now my C# programm calculates the position of the train in dependence of time and acceleration, gives back the appropriate framenumber and draw this frame.


Additional Information:

There is another project (not written by me) in C/C++ who uses DirectShow and the avcodec-LAV directly with a similar way I do and it works fine! Thats because I had the idea to use a codec / filter like the avrcodec-lav by myself. But I can't find an interop or interface to work with C#.



Thanks everyone for reading this and trying to help! :)

-------------Problems Reply------------

Obtaining specific frame by seeking filter graph (the entire pipeline) is pretty slow since every seek operation involves the following on its backyard: flushing everything, possibly re-creating worker threads, seeking to first key frame/splice point/clean point/I-Frame before the requested time, start of decoding starting from found position skipping frames until originally requested time is reached.

Overall, the method works well when you scrub paused video, or retrieve specific still frames. When however you try to play this as smooth video, it eventually causes significant part of the effort to be wasted and spent on seeking within video stream.

Solutions here are:

  • re-encode video to remove or reduce temporal compression (e.g. Motion JPEG AVI/MOV/MP4 files)
  • whenever possible prefer to skip frames and/or re-timestamp them according to your algorithm instead of seeking
  • have a cached of decoded video frames and pick from there, populate them as necessary in worker thread

The latter two are unfortunately hard to achieve without advanced filter development (where continuous decoding without interruption by seeking operations is the key to achieving decent performance). With basic DirectShow.Net you only have basic control over streaming and hence the first item from the list above.

Category:c# Views:2 Time:2018-12-15

Related post

Copyright (C) dskims.com, All Rights Reserved.

processed in 0.218 (s). 11 q(s)