Murga-Projects Forums
FLTK and ffmpeg - Printable Version

+- Murga-Projects Forums (http://www.murga-projects.com/forum)
+-- Forum: Project Forums (/forumdisplay.php?fid=1)
+--- Forum: MurgaLua - General (/forumdisplay.php?fid=2)
+--- Thread: FLTK and ffmpeg (/showthread.php?tid=312)


FLTK and ffmpeg - dvw86 - 04-06-2008 04:25 AM

Is it possible to capture video with ffmpeg and use murgaLua to display it on the screen? Videoview does this with fltk but I assume that they used some type of "C" language.

videoview


RE: FLTK and ffmpeg - mikshaw - 04-06-2008 11:21 AM

I'm sure videoview uses other libraries that are written specifically to decode and display video.

The only way I could think of to do this with the current mugaLua is to use external applications that are already built to perform these tasks. You could probably use a timeout to display a series of still frames to emulate video playback, but as far as capturing goes I'm pretty sure you'd need to outsource it to another application. Perhaps the library communication that John has been working on for the next release will be able to use some external library to capture?


RE: FLTK and ffmpeg - Juergen - 04-07-2008 01:06 AM

It doesn't make sense to en-/decode or grab video with Lua or any other similar language (perl,python,php,....)
These are pretty low level tasks which need a lot OS, hardware and userspace interaction. Therefore it would only make sense to write a GUI in that languages and use the available GUI bindings and for the video IO, decode and encode stuff use a low level C/C++ library.
Even if a high level library is used for video/audio en-/decoding, writing a player includes highly OS specific code which has to do a lot more then most people think (the video and audio stream has to be synced at all times, which isn't a trivial task ;-) Also it should still perform well. Therefore it is quite a hard task to program a good video player.

On Linux there are quite a few alternatives. There is mplayer, VLC, Xine and even ffmpeg comes with a rudimentary player (ffplay). Mplayer, VLC and Xine can also grab video from a V4L source. Also it is possible to script all three of them and let them render into a fltk window. So it is possible to use them as a backend to output video into a fltk window of a murgaLua script.

Btw.: One reason I wanted the fl_xid() function included in murgaLua was to embed mplayer in a script. Unfortunately to have that reliably working, there are still a few functions needed. The problem is that the std. Lua IO functions can't be used to reliably script mplayer over pipes, because read blocks, ... But VLC and Xine should work, because they can be scripted over a socket and with luasocket and copas it should be no problem to get that going.

Juergen


RE: FLTK and ffmpeg - dvw86 - 04-07-2008 02:31 AM

Juergen,
You have my attention. I would be quite happy to use xine and have it display in an FLTK window. I'll play around with that. Thanks


RE: FLTK and ffmpeg - Juergen - 04-07-2008 03:13 AM

dvw86 Wrote:
Juergen,
You have my attention. I would be quite happy to use xine and have it display in an FLTK window. I'll play around with that. Thanks


This is a primitive mplayer example. The problem is that with the primitive Lua IO model, I can't read the output from mplayer reliably, because I have no Idea how much lines are in the buffer and when I call read() on an empty buffer it will block. When this is solved, it would be no problem to easily script mplayer.

Code:
#!/usr/bin/env murgaLua
function my_cb(self)
        if not initialized then
                os.execute("mkfifo /tmp/mpfifo")
                mpout=io.popen("LANG=C mplayer -quiet -input file=/tmp/mpfifo -wid "..fltk.fl_xid(group).." "..arg[1].." &","r")
                mpin=io.open("/tmp/mpfifo","w")
                initialized=1
        end
        mpin:write("seek 10\n")
        mpin:flush()
end

w=fltk.Fl_Double_Window(640,510,"Mpayer Gui")
w:size_range(32,20,Fl:w(),Fl:h())
button1=fltk.Fl_Button(0,495,70,15,"Press Me!")
group=fltk.Fl_Window(0,0,640,480)
w:resizable(group)
button1:callback(my_cb)
w:show()
Fl:run()


Juergen


RE: FLTK and ffmpeg - mikshaw - 04-08-2008 12:20 AM

That's pretty cool.
I did a test with urxvt, and was suprised how simple it was.
Unfortunately, I have yet to find a reliable way to close both the terminal and the container window...so far i'm using if os.execute("killall urxvt")==0 then os.exit() end, but that's far from ideal. There doesn't seem to be a way to get the pid for that particular instance of urxvt from Lua.


RE: FLTK and ffmpeg - dvw86 - 04-08-2008 02:33 AM

This is all fun and new to me, so my first test was to try it without murgaLua and this is what I came up with to display my webcam in a xine window. The next step would be to embed the xine display into a fltk window.

Code:
#!/bin/sh
rm -f /tmp/stream
mkfifo /tmp/stream
xine -G 352x288 -B --no-splash fifo:///tmp/stream &
ffmpeg -vd /dev/video -f video4linux -r 25 -s 352x288 -sameq -f mpeg -y /tmp/stream


This works most of the time but it is not very reliable. I often get an empty or scrambled xine window that never displays the stream. When it does work there is a large (about 5 or 10 second) delay between what is captured and what is displayed. It is an encouraging start though.


RE: FLTK and ffmpeg - Juergen - 04-08-2008 03:27 AM

dvw86 Wrote:
This is all fun and new to me, so my first test was to try it without murgaLua and this is what I came up with to display my webcam in a xine window. The next step would be to embed the xine display into a fltk window.

Code:
#!/bin/sh
rm -f /tmp/stream
mkfifo /tmp/stream
xine -G 352x288 -B --no-splash fifo:///tmp/stream &
ffmpeg -vd /dev/video -f video4linux -r 25 -s 352x288 -sameq -f mpeg -y /tmp/stream


This works most of the time but it is not very reliable. I often get an empty or scrambled xine window that never displays the stream. When it does work there is a large (about 5 or 10 second) delay between what is captured and what is displayed. It is an encouraging start though.

Normally I don't use Xine, but mplayer, which should be able to do that whithout calling ffmpeg.
But to start with your example, you should try the following first:

Code:
ffmpeg -vd /dev/video -f video4linux -r 25 -s 352x288 -sameq -f mpeg - | xine -G 352x288 -B --no-splash stdin:/

It should work better, but I haven't tested it, since I don't have xine available at the moment.

Juergen


RE: FLTK and ffmpeg - Juergen - 04-08-2008 03:44 AM

mikshaw Wrote:
That's pretty cool.
I did a test with urxvt, and was suprised how simple it was.
Unfortunately, I have yet to find a reliable way to close both the terminal and the container window...so far i'm using if os.execute("killall urxvt")==0 then os.exit() end, but that's far from ideal. There doesn't seem to be a way to get the pid for that particular instance of urxvt from Lua.

Without proper suppoert for the posix API either through the posix module or alien every solution would be a hack.
But there is one thing you could do (also a hack):

Code:
pidpipe=io.popen("urxvt -embed "..fltk.fl_xid(...).."..... >/dev/null & echo $!","r")
urxvtpid=pidpipe:read("*l")
pidpipe:close()
pidpipe=nil
os.execute("kill "..pid)


I didn't test it, but it should work.

Juergen


RE: FLTK and ffmpeg - mikshaw - 04-08-2008 04:38 AM

Quote:
>/dev/null & echo $!"

Looks like this is where I was going wrong.
I'd tried "urxvt & && echo $!" and "urxvt &; echo $!", which choked, and "(urxvt &) && echo $!", which echoed nothing.
I didn't know using a single "&" would separate the commands, but that makes sense to me now.


RE: FLTK and ffmpeg - dvw86 - 04-08-2008 09:30 AM

Juergen Wrote:
Normally I don't use Xine, but mplayer, which should be able to do that whithout calling ffmpeg.
But to start with your example, you should try the following first:

Code:
ffmpeg -vd /dev/video -f video4linux -r 25 -s 352x288 -sameq -f mpeg - | xine -G 352x288 -B --no-splash stdin:/

It should work better, but I haven't tested it, since I don't have xine available at the moment.

Juergen


Well I tried that about ten or twenty times and it worked once. All the other times the camera would come on and ffmpeg would start encoding. The xine window would open and display a scrambled window. As soon as I stopped ffmpeg, the xine window would display it's normal splash screen. It makes me wonder if it is a xine or a ffmpeg issue. If mplayer is more repeatable, I guess that I will have to install it. That would be a bit of a bummer since xine works so well for me otherwise.


RE: FLTK and ffmpeg - Juergen - 04-08-2008 11:01 PM

mikshaw Wrote:

Quote:
>/dev/null & echo $!"

Looks like this is where I was going wrong.
I'd tried "urxvt & && echo $!" and "urxvt &; echo $!", which choked, and "(urxvt &) && echo $!", which echoed nothing.
I didn't know using a single "&" would separate the commands, but that makes sense to me now.


All control operators terminate a command. Control operators are "|" "&" "||" "&&" ";" ";;" "(" ")" and a new line.

You can also write someting like os.execute("cmd1 \n cmd2 \n cmd3") or use the ";" operator to separate two commands, but you can't use two control operators together (except for newline or the subshell operators). The || and && operators also require two expressions (expression1 || expression2) which cmd& can't be because it is executed in the background and can't give anything back.

Actually the bash manpage isn't that bad. If you open it and go down a few paragraphs to the "DEFINITIONS" section you'll see it is pretty easy. Although you should be aware that bash is only a bourne compatible shell and is a mix of Korn shell, Bourne shell and POSIX shell (which is also a mix). So not everything in that manual applies to other shells.

Juergen


RE: FLTK and ffmpeg - Juergen - 04-08-2008 11:18 PM

dvw86 Wrote:
Well I tried that about ten or twenty times and it worked once. All the other times the camera would come on and ffmpeg would start encoding. The xine window would open and display a scrambled window. As soon as I stopped ffmpeg, the xine window would display it's normal splash screen. It makes me wonder if it is a xine or a ffmpeg issue. If mplayer is more repeatable, I guess that I will have to install it. That would be a bit of a bummer since xine works so well for me otherwise.


Actually the problem might be on the Xine side of the pipeline but on the ffmpeg side. You should test it with files first. It would be a good to try something like that (bash syntax):

Code:
mkdir vout
for ((i=1;i<=20;i++))
  do
     ffmpeg -vd /dev/video -f video4linux -r 25 -s 352x288 -sameq -f mpeg -t 30 vout/clip`printf "%.2i" $i`.mpg
  done

echo vout/* | xargs xine

If that works well, then there might be a problem with the pipes, but the chance is higher that there is a problem with the video input.

Juergen


RE: FLTK and ffmpeg - JohnMurga - 04-09-2008 07:25 AM

Hi,

Because of this thread I have pondering on incorporating sys_proc, sys_file, sys_comm and maybe sys_env into 0.6.8.

The code is complete and I am trying to get time to do all the tests and cross builds, but I feel that sys_file in particular would help people here ... Juergen, your thoughts ?

Cheers
JohnM


RE: FLTK and ffmpeg - JohnMurga - 04-10-2008 06:53 AM

JohnMurga Wrote:
Hi,

Because of this thread I have pondering on incorporating sys_proc, sys_file, sys_comm and maybe sys_env into 0.6.8.

The code is complete and I am trying to get time to do all the tests and cross builds, but I feel that sys_file in particular would help people here ... Juergen, your thoughts ?

Cheers
JohnM


And what about luaEx (http://lua-ex.luaforge.net/) ?

LuaSys is proving to be a pain in the arse :-)

Cheers
JohnM


RE: FLTK and ffmpeg - Juergen - 04-10-2008 07:39 AM

JohnMurga Wrote:

JohnMurga Wrote:
Hi,

Because of this thread I have pondering on incorporating sys_proc, sys_file, sys_comm and maybe sys_env into 0.6.8.

The code is complete and I am trying to get time to do all the tests and cross builds, but I feel that sys_file in particular would help people here ... Juergen, your thoughts ?

Cheers
JohnM


And what about luaEx (http://lua-ex.luaforge.net/) ?

LuaSys is proving to be a pain in the arse :-)

Cheers
JohnM

Actually I have proposed luaEx a few days ago ;-)

Nevertheless it would be good to have an IO implementation that can use buffers, work with file descriptors, can manipulate them and provides a select/poll/epoll/... type call. Also pipes and something like spawn implementation in luaex would be a really good thing.
While I have no problem with luasys, it looks very raw and is completely undocumented.

Unfortunately on Linux I have the problem that I can't use most of the external modules, because I get a segmentation fault with murgaLua.
It would also be a good idea to export the symbols from libLua, which should remove the necessity to recompile libraries for murgaLua to explicitly link against liblua. (I tested it with the original lua interpreter and it works also with UPX). Unfortunately it doesn't work this way on Windows. So for Windows it will be necessary to either distribute a liblua or compile the external modules specifically for murgaLua.

Juergen


RE: FLTK and ffmpeg - JohnMurga - 04-10-2008 10:48 PM

luaEx doesn't have the IO stuff that I think you'd need ...

I have done a lot of work tidying up LuaSys and got it all working nicely on Windows last night ... If it works OK on Mac/Linux then we'll be looking good.

Primarily what you'll be getting is the IO stuff, memory buffers, serial port support, environment and and spawn implementations from LuaSys.

Fingers crossed for this evening (!)

If it all works then you are right ...
I have to document things, as LuaSys is completely undocumented :-)

Cheers
JohnM


RE: FLTK and ffmpeg - dvw86 - 04-11-2008 04:32 AM

JohnMurga Wrote:
Primarily what you'll be getting is the IO stuff, memory buffers, serial port support, environment and and spawn implementations from LuaSys.


WooHoo! Thanks! With serial port support I could use murgaLua to talk with PLC's.
PLC Info


RE: FLTK and ffmpeg - JohnMurga - 04-12-2008 05:01 AM

Juergen Wrote:
Unfortunately on Linux I have the problem that I can't use most of the external modules, because I get a segmentation fault with murgaLua.
It would also be a good idea to export the symbols from libLua, which should remove the necessity to recompile libraries for murgaLua to explicitly link against liblua. (I tested it with the original lua interpreter and it works also with UPX). Unfortunately it doesn't work this way on Windows. So for Windows it will be necessary to either distribute a liblua or compile the external modules specifically for murgaLua.


I think that is due to my link options under Linux, it'll most likely work on 0.6.8 ... Either for Windows I have also put an example in place.

Either way you'll get to try it out this weekend as I'll definitely do the release in the next couple of days ...

Cheers
JohnM