How to execute Python script from C code using EXECL?

I would like to know how I can execute a Python (or Lua etc) script from my C code using execl (or similar)?

The following is some "parent / child" code showing how I am sending a STREAM of data to the child using PIPES. The code may not be perfect but you get the idea. Note the execl at the bottom:

#include <sys/types.h> #include <stdio.h> #include <stdlib.h> #include <string.h> #include <unistd.h> #define STDIN_FILENO 0 /* Standard input. */ #define STDOUT_FILENO 1 /* Standard output. */ #define STDERR_FILENO 2 /* Standard error output. */ #define MAXLINE 4096 int main(void){ int n, parent_child_pipe[2], child_parent_pipe[2]; pid_t pid; char line[MAXLINE]; if (pipe(parent_child_pipe) < 0 || pipe(child_parent_pipe) < 0) puts("Error creating pipes...\n"); if ( (pid = fork()) < 0) puts("Error forking...\n"); else if (pid > 0) { /* PARENT */ close(parent_child_pipe[0]); close(child_parent_pipe[1]); while (fgets(line, MAXLINE, stdin) != NULL) { n = strlen(line); if (write(parent_child_pipe[1], line, n) != n) puts("write error to pipe...\n"); if ( (n = read(child_parent_pipe[0], line, MAXLINE)) < 0) puts("read error from pipe...\n"); if (n == 0) { puts("child closed pipe...\n"); break; } line[n] = 0; /* null terminate */ if (fputs(line, stdout) == EOF) puts("fputs error...\n"); } if (ferror(stdin)) puts("fgets error on stdin...\n"); exit(0); } else { /* CHILD */ close(parent_child_pipe[1]); close(child_parent_pipe[0]); if (parent_child_pipe[0] != STDIN_FILENO) { if (dup2(parent_child_pipe[0], STDIN_FILENO) != STDIN_FILENO) puts("dup2 error to stdin...\n"); close(parent_child_pipe[0]); } if (child_parent_pipe[1] != STDOUT_FILENO) { if (dup2(child_parent_pipe[1], STDOUT_FILENO) != STDOUT_FILENO) puts("dup2 error to stdout...\n"); close(child_parent_pipe[1]); } **if (execl("./child", "child", (char *) 0) < 0)** puts("execl error...\n"); } }

Now the "child" program above is written in C and simply receives the stream via STDIN, manipulates the stream, and sends it back out using STDOUT.

Something like:

#include <stdio.h> #include <stdlib.h> #include <string.h> #include <unistd.h> #define STDIN_FILENO 0 /* Standard input. */ #define STDOUT_FILENO 1 /* Standard output. */ #define STDERR_FILENO 2 /* Standard error output. */ #define MAXLINE 4096 int main(void){ int n; char line[MAXLINE]; while ( (n = read(STDIN_FILENO, line, MAXLINE)) > 0) { line[n] = 0; /* null terminate */ n = strlen(line); if (write(STDOUT_FILENO, line, n) != n) puts("write error"); } exit(0); }

So that is working fine, but now I want to be able to write the CHILD in any scripting language like Python / Lua etc. How can I do this? I have tried stuff like:

if (execl("path to python", "test.py", (char *) 0) < 0)

But it just seems to HANG waiting to receive input?

Can someone please help me on this? I assume PIPES can talk to anything like Lua / Python that can read from STDIN and send back to STDOUT?

UPDATE:

I have made some small changes now, here is the Python file: "NullFilter.py" that simply ECHOS what it is sent:

#!/usr/bin/python import sys class NullFilter: def execute(self): #Read data from STDIN... data = sys.stdin.read() #Write data to STDOUT... sys.stdout.write(data) exit(0) if __name__ == '__main__': nf = NullFilter() nf.execute()

And now the C code calls it using:

... if (execl("/usr/bin/python","./NullFilter.py","./NullFilter.py",NULL, (char *) 0) < 0) puts("execl error...\n"); ...

When I run it now I can enter text into STDIN, but have to hit CRTL-C to see what has happened: Here is the result:

[email protected]:~/Desktop/pipe example$ ./parent hello hello again ^CTraceback (most recent call last): File "./NullFilter.py", line 17, in <module> nf.execute() File "./NullFilter.py", line 10, in execute [email protected]:~/Desktop/pipe example$ data = sys.stdin.read() KeyboardInterrupt [email protected]:~/Desktop/pipe example$

UPDATE 2:

OK, so I have been playing around a little, and simply changed the Python code from "sys.stdin.read()" to "sys.stdin.readline()" and it seems to work to a certain degree, BUT NOT PERFECT at all...

#!/usr/bin/python import sys class NullFilter: def execute(self): #Read data from STDIN... data = "" for line in sys.stdin.readline(): data = data + line #Write data to STDOUT... sys.stdout.write(data) exit(0) if __name__ == '__main__': nf = NullFilter() nf.execute()

This is a workaround BUT not perfect at all. Any other ideas how I can get the STDIN to read the stream UNBUFFERED? I have looked at the SELECT module in Python:

http://docs.python.org/library/select.html

I have also tried passing "-u" to Python to make it "unbuffered" but still no luck ;-(

But surely this cannot be so difficult?

Lynton

-------------Problems Reply------------

After much playing around and trying to flush everything I realised I needed to CLOSE the STDOUT end of the PIPE going from the parent to the child (after writing to the pipe of course)...

So the code is now:

#include <sys/types.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>

#define STDIN_FILENO 0 /* Standard input. */
#define STDOUT_FILENO 1 /* Standard output. */
#define STDERR_FILENO 2 /* Standard error output. */
#define MAXLINE 4096

int main(void){
int n, parent_child_pipe[2], child_parent_pipe[2];
pid_t pid;
char line[MAXLINE];
int rv;

if (pipe(parent_child_pipe) < 0 || pipe(child_parent_pipe) < 0)
puts("Error creating pipes...\n");

if ( (pid = fork()) < 0)
puts("Error forking...\n");
else if (pid > 0) { /* PARENT */
close(parent_child_pipe[0]);
close(child_parent_pipe[1]);
while (fgets(line, MAXLINE, stdin) != NULL) {
n = strlen(line);
if (write(parent_child_pipe[1], line, n) != n)
puts("write error to pipe...\n");
close(parent_child_pipe[1]);
wait(&rv);
if ( (n = read(child_parent_pipe[0], line, MAXLINE)) < 0)
puts("read error from pipe...\n");
if (n == 0) {
puts("child closed pipe...\n");
break;
}
line[n] = 0; /* null terminate */
if (fputs(line, stdout) == EOF)
puts("fputs error...\n");
}
if (ferror(stdin))
puts("fgets error on stdin...\n");
exit(0);

} else { /* CHILD */
close(parent_child_pipe[1]);
close(child_parent_pipe[0]);
if (parent_child_pipe[0] != STDIN_FILENO) {
if (dup2(parent_child_pipe[0], STDIN_FILENO) != STDIN_FILENO)
puts("dup2 error to stdin...\n");
close(parent_child_pipe[0]);
}
if (child_parent_pipe[1] != STDOUT_FILENO) {
if (dup2(child_parent_pipe[1], STDOUT_FILENO) != STDOUT_FILENO)
puts("dup2 error to stdout...\n");
close(child_parent_pipe[1]);
}
if (execl("./NullFilter.py", "./NullFilter.py", (char *) 0) < 0)
puts("execl error...\n");
}
}

You can see the "close(parent_child_pipe[1]);" just after writing to the PIPE above, that is the crucial piece I had to do. That will force the stream to be flushed to the Python script, Lua script , C code etc....

In the above you will see I am executing a Python script "NullFilter.py"....

NOTE: If you run the code above it will work for one iteration of input / output as the Python script closes the pipe after the first test, but the essentials are there for you to build on...

Thanks for all the help though, I have learnt a lot from this exercise ;-)

Lynton

I believe you can achieve what you want to do by starting the python file with the shebang line like this:

#!/usr/bin/python

and ensuring it is executable, then using the script as the executable field in execl()

See http://linux.about.com/od/commands/l/blcmdl2_execve.htm:

Filename must be either a binary executable, or a script starting with a line of the form "#! interpreter [arg]". In the latter case, the interpreter must be a valid pathname for an executable which is not itself a script, which will be invoked as interpreter [arg] filename.

In the case of passing arguments (also theoretically valid), it doesn't look like python is actually executing the script and finishing as it should - rather it is opening an interactive terminal which is why it hangs - it wants you to communicate via stdin. Try the absolute path to "test.py" and see what happens, as it sounds like python is unable to find it.

There is, actually, another method. There is no reason you can't pass the script line-by-line to the interpreter via pipes and run it that way.

Did You try a flush on the child ?

UPDATE:

I tried with your C code and with these 2 scripts (perl and python) and their behavior is exactly like the C child on my machine

#!/usr/bin/perl -w
use IO::Handle;

while (<>)
{
print;
flush STDOUT;
}

And

#!/usr/bin/python
import sys

class NullFilter:
def execute(self):
while True:
line=sys.stdin.readline()
if not line: break
sys.stdout.write(line)
sys.stdout.flush()
exit(0)
if __name__ == '__main__':
nf = NullFilter()
nf.execute()

Category:c# Views:0 Time:2010-09-06
Tags: c#

Related post

  • python and django: execute python script from django web form? 2011-02-12

    I have fully functional python program which I run from shell. All arguments and options are supplied using option parser. What I would like to do is to create simple web interface for this program using django. I am stacked on transferring values re

  • Execute python script inside a python script 2010-08-05

    I have a scenario where i want to dynamically generate a python script - inside my main python script - store it as a string and then when need be, execute this dynamically generated script from my main script. Is this possible, if so how? thanks ---

  • How to execute Python script from CreateProcess in C on Windows? 2010-09-09

    I have managed to get C code calling Python scripts happily on Unix using PIPES within the C code. I now need to do the same on Windows. Essentially I would like to write scripts in different scripting languages like Python / Lua etc on Windows and b

  • Problem with using system call to execute python script from MATLAB 2011-02-15

    I have a MATLAB function that needs to communicate (not rapidly, and not often) with python code. I have MATLAB write numbers to a file. Python reads the file, does some calculations, and writes some results to another file. MATLAB then reads that fi

  • Executing Python script from Java in Tomcat project 2011-06-30

    I want to execute a Python script (including Scipy and Numpy) from Java which should be bundled with the tomcat project by eclipse. Calling Python from Java is trivial: Runtime.getRuntime().exec("python ..."); How can I achieve bundling the script to

  • Execute python Script on Crontab 2012-01-04

    I'm trying to execute a python script using the linux crontab, but i found a lot of solution and anyone works, for example edit the anacron at /etc/cron.d ou use crontab -e. I want to run this script every 10 minutes. What file must I edit to configu

  • Executing Python Script From Command Line is Hiding Print Statements 2013-03-21

    I know this must be a super basic question, however, I have tried finding a simple answer throughout SO and cannot find one. So my question is this: How can I execute a python script from the command line such that I can see print statements. For exa

  • Going nuts with executing python script via crontab on debian! 2009-04-03

    This is what my crontab file looks like: * * * * * root /usr/bin/python /root/test.py >> /root/classwatch.log 2>&1 This is what my python script looks like: #!/usr/bin/python print "hello" The cronjob creates the log file. But it is empt

  • Problem while running python script in java code 2009-08-21

    When i run a python script from the below java code, where an input file is given as an argument to the python script as well as an "-v" option, i get a IOException String pythonScriptPath="\"C:\\Program Files\\bin\\CsvFile.py\""; String Filepath="C:

  • Escaping Bash wildcards when executing python script from bash script? 2011-02-27

    I've got a bash script that defines an array of file globs containing * wildcards. The script executes a python script passing each of these globs as a parameter. On the command line I can quote the entire glob, and python is happy with it. In the ba

  • Executing Shell script from Java code 2011-03-10

    I have the gui which have the button as "mount to server"....am having the ubuntu client and server system.....When I click the mount button ,it should mount to server folder... for example, one folder name "OUTPUT" is in the server..in client machin

  • launchd executes python script, but import fails 2011-04-06

    I wrote a python script using appscript to track my currently active window. I am running it via launchd, but when I do that it can't import appscript. I have set the PYTHONPATH in the plist for launchd, but I think launchd is not reading .pth files

  • Executing python scripts with subprocess.call using shebang 2011-06-22

    I'm writing a (somewhat) modular application in Python 3 and I'd like to run arbitrary programs from it, said program being specified at runtime and not necessarily a python script. So I use for example, subprocess.call([spam, "-i", eggs, "-o", ham])

  • how to execute a script using java code at runtime 2011-07-07

    I have a Java application and a NSIS script. I need to run the NSIS script via Java code at runtime. I know the following code is used for runtime execution. Process p = Runtime.getRuntime().exec(); My problem is i don't know how to execute my script

  • Call python script within java code (runtime.exec) 2011-09-16

    I'm trying to run a python script in java but I'm having some troubles. I'm using the command bellow to execute the python script which is inside a folder called python in my java project: Runtime r = Runtime.getRuntime(); Process p = r.exec("cmd /c

  • execute python script with function from command line, Linux 2011-10-07

    I have my python file called convertImage.py and inside the file I have a script that converts an image to my liking, the entire converting script is set inside a function called convertFile(fileName) Now my problem is I need to execute this python s

  • execute python script when directory is not empty - Directory Monitoring 2011-10-10

    I have a python script that converts images and videos withing a directory. The problem, the python script executes manually but I need the script to execute automatically when a file is dropped into the directory under a linux platform. What would b

  • git cannot execute python-script as hook 2012-02-16

    I have created a little pre-commit hook in python. This hook works like a charm under Linux, but in Windows it keeps telling me: error: cannot spawn .git/hooks/pre-commit: No such file or directory I know there have been similar questions here about

  • Execute python-script via Url 2012-03-05

    Is it possible to execute a python-script on a server without using something like django? I mean I put script.py on host.com and want to call it like this: http://www.host.com/script.py The script then does something like calculating some variables

Copyright (C) dskims.com, All Rights Reserved.

processed in 0.125 (s). 11 q(s)