ReadFile hangs on reading from child process.
in [Win32 Kernel]
|
Prev: How are Windows Connected to Processes?
Next: Apple Sues the HTC (Android phone maker) with Patents that effectsus all
From: IlyaK on 2 Mar 2010 14:48 Hello: Strange hang problem with ReadFile. Here is the algorithm: The idea is to launch a python script from one Win32 GUI application, then read its std output and store it in a file. Do this until the script is done. 1. I create a child process using CreateProcess with the code pretty much copied from http://support.microsoft.com/default.aspx?scid=kb;en-us;190351. However, instead of creating some special child process I just launch a python script. 2. Then start reading from the hChildStdOut using ReadFile. 3. The problem is that ReadFile at the end hangs indefinitely waiting for more data from the child process. Is there any way to know if the child process is done (or dead) and stop reading. I tried several methods to do that, however, they weren't reliable so far. 1. Check if the ReadFile lpNumberOfBytesRead is smaller than nNumberOfBytesToRead (http://msdn.microsoft.com/en-us/library/ aa365467%28VS.85%29.aspx). This didn't work because at the beginning sometimes the Python script returns lpNumberOfBytesRead < nNumberOfBytesToRead. It does some init and it takes time, I suppose. 2. Use GetExitCodeProcess (http://msdn.microsoft.com/en-us/library/ ms683189%28VS.85%29.aspx) on the child process handle. This method didn't work either. For some reason the GetExitCodeProcess would indicate that the process is still active. Please help. -Ilya.
From: Hector Santos on 2 Mar 2010 15:40 You should show code because that first link with redirection logic should work for you. However, there are ways do this without using pipes. This is pretty standard practice coding stuff. So show code to see where you went wrong. -- IlyaK wrote: > Hello: > > Strange hang problem with ReadFile. > Here is the algorithm: > > The idea is to launch a python script from one Win32 GUI application, > then read its std output and store it in a file. Do this until the > script is done. > 1. I create a child process using CreateProcess with the code pretty > much copied from http://support.microsoft.com/default.aspx?scid=kb;en-us;190351. > However, instead of creating some special child process I just launch > a python script. > 2. Then start reading from the hChildStdOut using ReadFile. > 3. The problem is that ReadFile at the end hangs indefinitely waiting > for more data from the child process. > > Is there any way to know if the child process is done (or dead) and > stop reading. I tried several methods to do that, however, they > weren't reliable so far. > 1. Check if the ReadFile lpNumberOfBytesRead is smaller than > nNumberOfBytesToRead (http://msdn.microsoft.com/en-us/library/ > aa365467%28VS.85%29.aspx). This didn't work because at the beginning > sometimes the Python script returns lpNumberOfBytesRead < > nNumberOfBytesToRead. It does some init and it takes time, I suppose. > 2. Use GetExitCodeProcess (http://msdn.microsoft.com/en-us/library/ > ms683189%28VS.85%29.aspx) on the child process handle. This method > didn't work either. For some reason the GetExitCodeProcess would > indicate that the process is still active. > > Please help. > -Ilya. -- HLS
From: IlyaK on 2 Mar 2010 16:05 Fair enough. Here is the code: { // start the child process HANDLE hChildStdOut; HANDLE hChildStdIn; HANDLE hChildStdErr; HANDLE hChildProc; if (!launch_redirected_child(m_testCmdLine.c_str(), &hChildStdOut, &hChildStdIn, &hChildStdErr, &hChildProc)){ printf("Error creating child\n"); return -1; } // Wait for the script to finish its busyness, // getting the output and checking the cancel // flag const int iBufferSize = 256; char * readBuffer = new char[iBufferSize]; memset(readBuffer, 0, iBufferSize); int numRead = 0; unsigned long lErrors = 0; long isEof = 0; printf("BEGIN Log:\n"); if (m_bDoScriptLog == true) { while (get_cancel_test() != true) { if ( (numRead = pipe_read( hChildStdOut, readBuffer, iBufferSize )) > 0 ) { printf("Log: %d\n", numRead); printf("Log: %s\n", string(readBuffer, numRead).c_str()); GetExitCodeProcess(hChildProc, &lErrors); printf("Log: ", lErrors); if (lErrors != STILL_ACTIVE && numRead < iBufferSize) { printf("Log: Exit"); break; } } else { // Check if the pipe is broken, then exit, // Otherwise report error, and aslo exit printf("Exit on Error\n"); break; } //memset(readBuffer, 0, iBufferSize); } } printf("END Log.\n"); delete [] readBuffer; status = CloseHandle(hChildStdOut); status = CloseHandle(hChildStdIn); status = CloseHandle(hChildStdErr); status = CloseHandle(hChildProc); } /** * The function will set up STARTUPINFO structure, and launch redirected child. * It will return the handles to talk to this child process. * Make sure that the handles are properly closed after use. * Lifted from http://support.microsoft.com/default.aspx?scid=kb;en-us;190351 * * @param command * @param hChildStdOut * @param hChildStdIn * @param hChildStdErr * * @return 0 - error occured. Use GetLastError to figure out what happened. */ int launch_redirected_child(const char * command, HANDLE * hChildStdOut, HANDLE * hChildStdIn, HANDLE * hChildStdErr, HANDLE * hChildProcess) { HANDLE hOutputReadTmp,hOutputRead,hOutputWrite; HANDLE hInputWriteTmp,hInputRead,hInputWrite; SECURITY_ATTRIBUTES sa; PROCESS_INFORMATION pi; STARTUPINFO si; // Set up the security attributes struct. sa.nLength= sizeof(SECURITY_ATTRIBUTES); sa.lpSecurityDescriptor = NULL; sa.bInheritHandle = TRUE; // Create the child output pipe - child sends. if (!CreatePipe(&hOutputReadTmp,&hOutputWrite,&sa,0)) { return 0; } // Create a duplicate of the output write handle for the std error // write handle. This is necessary in case the child application // closes one of its std output handles. if (!DuplicateHandle(GetCurrentProcess(),hOutputWrite, GetCurrentProcess(), hChildStdErr, 0,TRUE, DUPLICATE_SAME_ACCESS)) { return 0; } // Create the child input pipe - child receives. if (!CreatePipe(&hInputRead,&hInputWriteTmp,&sa,0)) { return 0; } // Create new output read handle and the input write handles. Set // the Properties to FALSE. Otherwise, the child inherits the // properties and, as a result, non-closeable handles to the pipes // are created. if (!DuplicateHandle(GetCurrentProcess(),hOutputReadTmp, GetCurrentProcess(), hChildStdOut, // Address of new handle. 0,FALSE, // Make it uninheritable. DUPLICATE_SAME_ACCESS)) { return 0; } if (!DuplicateHandle(GetCurrentProcess(),hInputWriteTmp, GetCurrentProcess(), hChildStdIn, // Address of new handle. 0,FALSE, // Make it uninheritable. DUPLICATE_SAME_ACCESS)) { return 0; } // Close inheritable copies of the handles you do not want to be // inherited. if (!CloseHandle(hOutputReadTmp)) { return 0; } if (!CloseHandle(hInputWriteTmp)) { return 0; } // Set up the start up info struct. ZeroMemory(&si,sizeof(STARTUPINFO)); si.cb = sizeof(STARTUPINFO); si.dwFlags = STARTF_USESTDHANDLES; si.hStdOutput = hOutputWrite; si.hStdInput = hInputRead; si.hStdError = *hChildStdErr; // Use this if you want to hide the child: // si.wShowWindow = SW_HIDE; // Note that dwFlags must include STARTF_USESHOWWINDOW if you want to // use the wShowWindow flags. // Launch the process that you want to redirect (in this case, // Child.exe). Make sure Child.exe is in the same directory as // redirect.c launch redirect from a command line to prevent location // confusion. if (!CreateProcess(NULL,(LPCWSTR)(command), NULL,NULL,TRUE, 0,NULL,NULL,&si,&pi)) { return 0; } // Close any unnecessary handles. if (!CloseHandle(pi.hThread)) { return 0; } if (hChildProcess != NULL) { *hChildProcess = pi.hProcess; } else { if (!CloseHandle(pi.hProcess)) { return 0; } } if (!CloseHandle(hOutputWrite)) { return 0; } if (!CloseHandle(hInputRead)) { return 0; } return 1; } int pipe_read (HANDLE fd, void *buffer, size_t count) { DWORD numRead; if (!ReadFile (fd, buffer, (DWORD)count, &numRead, NULL)) { //non overlapped return -1; } return (int) numRead; } On Mar 2, 12:40 pm, Hector Santos <sant9...(a)nospam.gmail.com> wrote: > You should show code because that first link with redirection logic > should work for you. However, there are ways do this without using pipes. > > This is pretty standard practice coding stuff. So show code to see > where you went wrong. > > -- > > > > IlyaK wrote: > > Hello: > > > Strange hang problem with ReadFile. > > Here is the algorithm: > > > The idea is to launch a python script from one Win32 GUI application, > > then read its std output and store it in a file. Do this until the > > script is done. > > 1. I create a child process using CreateProcess with the code pretty > > much copied fromhttp://support.microsoft.com/default.aspx?scid=kb;en-us;190351. > > However, instead of creating some special child process I just launch > > a python script. > > 2. Then start reading from the hChildStdOut using ReadFile. > > 3. The problem is that ReadFile at the end hangs indefinitely waiting > > for more data from the child process. > > > Is there any way to know if the child process is done (or dead) and > > stop reading. I tried several methods to do that, however, they > > weren't reliable so far. > > 1. Check if the ReadFile lpNumberOfBytesRead is smaller than > > nNumberOfBytesToRead (http://msdn.microsoft.com/en-us/library/ > > aa365467%28VS.85%29.aspx). This didn't work because at the beginning > > sometimes the Python script returns lpNumberOfBytesRead < > > nNumberOfBytesToRead. It does some init and it takes time, I suppose. > > 2. Use GetExitCodeProcess (http://msdn.microsoft.com/en-us/library/ > > ms683189%28VS.85%29.aspx) on the child process handle. This method > > didn't work either. For some reason the GetExitCodeProcess would > > indicate that the process is still active. > > > Please help. > > -Ilya. > > -- > HLS
From: Hector Santos on 2 Mar 2010 17:40 You will probably get much input here, but it seems like you are prematurely closing the child process handle. You didn't show whats in the get_cancel_text() but that should be a kernel wait w/ timeout as you detect some cancel idea. You might be blocking here. If you don't want interactive I/O and just want to capture output, you really don't need piping. But it will work the same. The loop should be something like this example console program: ------------- CUT HERE ------------ // File: d:\wc5beta\testspawn4.cpp // Author: Hector Santos, Santronics Software, Inc. // Compile: cl testspawn4.cpp /MT #include <stdio.h> #include <afx.h> /****************************************************************** * SendToParent() is called by the redirection logic to that you * send the data as a message event to your parent window. In this * simple example we are just displaying it to the screen. ******************************************************************/ void SendToParent(const char *szBuf, const DWORD size) { printf("%s",szBuf); } /****************************************************************** * BOOL Run(UseDos, szCmd) * * Spawn a child process and capture the redirect output. * * UseDos = Set TRUE to use OS command interpreter * cmd = command to run (if DOS command, set UseDos=TRUE) * * return TRUE if successful. Otherwise FALSE, read extended error. ******************************************************************/ BOOL Run(const char *szCmd, const BOOL UseDos = FALSE) { SECURITY_ATTRIBUTES sa; ZeroMemory(&sa, sizeof(sa)); sa.nLength = sizeof(sa); sa.bInheritHandle = TRUE; HANDLE hOut = INVALID_HANDLE_VALUE; HANDLE hRedir = INVALID_HANDLE_VALUE; // Create Temp File for redirection char szTempPath[MAX_PATH]; char szOutput[MAX_PATH]; GetTempPath(sizeof(szTempPath),szTempPath); GetTempFileName(szTempPath, "tmp", 0, szOutput); // setup redirection handles // output handle must be WRITE mode, share READ // redirect handle must be READ mode, share WRITE hOut = CreateFile(szOutput, GENERIC_WRITE, FILE_SHARE_READ, &sa, CREATE_ALWAYS, FILE_ATTRIBUTE_TEMPORARY, 0); if (hOut == INVALID_HANDLE_VALUE) { return FALSE; } hRedir = CreateFile(szOutput, GENERIC_READ, FILE_SHARE_WRITE, 0, OPEN_EXISTING, 0, 0); if (hRedir == INVALID_HANDLE_VALUE) { CloseHandle(hOut); return FALSE; } // setup startup info, set std out/err handles // hide window STARTUPINFO si; ZeroMemory(&si, sizeof(si)); si.cb = sizeof(si); if (hOut != INVALID_HANDLE_VALUE) { si.dwFlags |= STARTF_USESTDHANDLES; si.hStdOutput = hOut; si.hStdError = hOut; si.wShowWindow = SW_HIDE; } // use the current OS comspec for DOS commands, i.e., DIR char cmd[MAX_PATH] = {0}; if (UseDos) { strcpy(cmd,getenv("comspec")); strcat(cmd," /c "); } strcat(cmd,szCmd); printf("-Process: %s\n",cmd); PROCESS_INFORMATION pi; ZeroMemory(&pi, sizeof(pi)); DWORD dwFlags = CREATE_NO_WINDOW; // NT/2000 only if (!CreateProcess(NULL, (char *)cmd, NULL, NULL, TRUE, dwFlags, NULL, NULL, &si, &pi)) { int err = GetLastError(); // preserve the CreateProcess error if (hOut != INVALID_HANDLE_VALUE) { CloseHandle(hOut); CloseHandle(hRedir); } SetLastError(err); return FALSE; } // let go of the child thread printf("-Waiting\n"); CloseHandle(pi.hThread); // wait for process to finish, capture redirection and // send it to the parent window/console DWORD dw; char buf[256]; do { ZeroMemory(&buf,sizeof(buf)); while (ReadFile(hRedir, &buf, sizeof(buf)-1, &dw, NULL)) { if (dw == 0) break; SendToParent(buf,dw); ZeroMemory(&buf,sizeof(buf)); } } while (WaitForSingleObject(pi.hProcess, 0) != WAIT_OBJECT_0); // perform any final flushing while (ReadFile(hRedir, &buf, sizeof(buf)-1, &dw, NULL)) { if (dw == 0) break; SendToParent(buf,dw); ZeroMemory(&buf,sizeof(buf)); } WaitForSingleObject(pi.hProcess, INFINITE); CloseHandle(pi.hProcess); CloseHandle(hOut); CloseHandle(hRedir); DeleteFile(szOutput); printf("-finished\n"); return TRUE; } void main(char argc, char *argv[]) { //------------------------------------------ // example #2 - using direct //------------------------------------------ if (!Run("dir *.cpp", TRUE)) { printf("Error %d\n",GetLastError()); return; } printf("-done\n"); } ---------------- CUT HERE ---------------- You can add a sanity/cancel check in the loop. Just make sure its not blocking. IlyaK wrote: > Fair enough. > Here is the code: > > { > // start the child process > HANDLE hChildStdOut; > HANDLE hChildStdIn; > HANDLE hChildStdErr; > HANDLE hChildProc; > if (!launch_redirected_child(m_testCmdLine.c_str(), &hChildStdOut, > &hChildStdIn, &hChildStdErr, &hChildProc)){ > printf("Error creating child\n"); > return -1; > } > > > // Wait for the script to finish its busyness, > // getting the output and checking the cancel > // flag > const int iBufferSize = 256; > char * readBuffer = new char[iBufferSize]; > memset(readBuffer, 0, iBufferSize); > int numRead = 0; > unsigned long lErrors = 0; > long isEof = 0; > > printf("BEGIN Log:\n"); > if (m_bDoScriptLog == true) { > while (get_cancel_test() != true) { > if ( (numRead = pipe_read( hChildStdOut, readBuffer, > iBufferSize )) > 0 ) { > printf("Log: %d\n", numRead); > printf("Log: %s\n", string(readBuffer, > numRead).c_str()); > > GetExitCodeProcess(hChildProc, &lErrors); > printf("Log: ", lErrors); > > if (lErrors != STILL_ACTIVE && numRead < iBufferSize) > { > printf("Log: Exit"); > break; > } > } else { > // Check if the pipe is broken, then exit, > // Otherwise report error, and aslo exit > printf("Exit on Error\n"); > break; > } > //memset(readBuffer, 0, iBufferSize); > } > } > printf("END Log.\n"); > > delete [] readBuffer; > > status = CloseHandle(hChildStdOut); > status = CloseHandle(hChildStdIn); > status = CloseHandle(hChildStdErr); > status = CloseHandle(hChildProc); > } > > /** > * The function will set up STARTUPINFO structure, and launch > redirected child. > * It will return the handles to talk to this child process. > * Make sure that the handles are properly closed after use. > * Lifted from http://support.microsoft.com/default.aspx?scid=kb;en-us;190351 > * > * @param command > * @param hChildStdOut > * @param hChildStdIn > * @param hChildStdErr > * > * @return 0 - error occured. Use GetLastError to figure out what > happened. > */ > int > launch_redirected_child(const char * command, > HANDLE * hChildStdOut, > HANDLE * hChildStdIn, > HANDLE * hChildStdErr, > HANDLE * hChildProcess) > { > HANDLE hOutputReadTmp,hOutputRead,hOutputWrite; > HANDLE hInputWriteTmp,hInputRead,hInputWrite; > SECURITY_ATTRIBUTES sa; > PROCESS_INFORMATION pi; > STARTUPINFO si; > > // Set up the security attributes struct. > sa.nLength= sizeof(SECURITY_ATTRIBUTES); > sa.lpSecurityDescriptor = NULL; > sa.bInheritHandle = TRUE; > > // Create the child output pipe - child sends. > if (!CreatePipe(&hOutputReadTmp,&hOutputWrite,&sa,0)) { > return 0; > } > > // Create a duplicate of the output write handle for the std error > // write handle. This is necessary in case the child application > // closes one of its std output handles. > if (!DuplicateHandle(GetCurrentProcess(),hOutputWrite, > GetCurrentProcess(), > hChildStdErr, > 0,TRUE, > DUPLICATE_SAME_ACCESS)) { > return 0; > } > > // Create the child input pipe - child receives. > if (!CreatePipe(&hInputRead,&hInputWriteTmp,&sa,0)) { > return 0; > } > > // Create new output read handle and the input write handles. Set > // the Properties to FALSE. Otherwise, the child inherits the > // properties and, as a result, non-closeable handles to the pipes > // are created. > if (!DuplicateHandle(GetCurrentProcess(),hOutputReadTmp, > GetCurrentProcess(), > hChildStdOut, // Address of new handle. > 0,FALSE, // Make it uninheritable. > DUPLICATE_SAME_ACCESS)) { > return 0; > } > > if (!DuplicateHandle(GetCurrentProcess(),hInputWriteTmp, > GetCurrentProcess(), > hChildStdIn, // Address of new handle. > 0,FALSE, // Make it uninheritable. > DUPLICATE_SAME_ACCESS)) { > return 0; > } > > // Close inheritable copies of the handles you do not want to be > // inherited. > if (!CloseHandle(hOutputReadTmp)) { > return 0; > } > if (!CloseHandle(hInputWriteTmp)) { > return 0; > } > > // Set up the start up info struct. > ZeroMemory(&si,sizeof(STARTUPINFO)); > si.cb = sizeof(STARTUPINFO); > si.dwFlags = STARTF_USESTDHANDLES; > si.hStdOutput = hOutputWrite; > si.hStdInput = hInputRead; > si.hStdError = *hChildStdErr; > // Use this if you want to hide the child: > // si.wShowWindow = SW_HIDE; > // Note that dwFlags must include STARTF_USESHOWWINDOW if you want > to > // use the wShowWindow flags. > > // Launch the process that you want to redirect (in this case, > // Child.exe). Make sure Child.exe is in the same directory as > // redirect.c launch redirect from a command line to prevent > location > // confusion. > if (!CreateProcess(NULL,(LPCWSTR)(command), > NULL,NULL,TRUE, > 0,NULL,NULL,&si,&pi)) { > return 0; > } > > // Close any unnecessary handles. > if (!CloseHandle(pi.hThread)) { > return 0; > } > if (hChildProcess != NULL) { > *hChildProcess = pi.hProcess; > } else { > if (!CloseHandle(pi.hProcess)) { > return 0; > } > } > if (!CloseHandle(hOutputWrite)) { > return 0; > } > if (!CloseHandle(hInputRead)) { > return 0; > } > return 1; > } > > int > pipe_read (HANDLE fd, void *buffer, size_t count) { > DWORD numRead; > if (!ReadFile (fd, buffer, (DWORD)count, &numRead, NULL)) > { //non overlapped > return -1; > } > return (int) numRead; > } > > On Mar 2, 12:40 pm, Hector Santos <sant9...(a)nospam.gmail.com> wrote: >> You should show code because that first link with redirection logic >> should work for you. However, there are ways do this without using pipes. >> >> This is pretty standard practice coding stuff. So show code to see >> where you went wrong. >> >> -- >> >> >> >> IlyaK wrote: >>> Hello: >>> Strange hang problem with ReadFile. >>> Here is the algorithm: >>> The idea is to launch a python script from one Win32 GUI application, >>> then read its std output and store it in a file. Do this until the >>> script is done. >>> 1. I create a child process using CreateProcess with the code pretty >>> much copied fromhttp://support.microsoft.com/default.aspx?scid=kb;en-us;190351. >>> However, instead of creating some special child process I just launch >>> a python script. >>> 2. Then start reading from the hChildStdOut using ReadFile. >>> 3. The problem is that ReadFile at the end hangs indefinitely waiting >>> for more data from the child process. >>> Is there any way to know if the child process is done (or dead) and >>> stop reading. I tried several methods to do that, however, they >>> weren't reliable so far. >>> 1. Check if the ReadFile lpNumberOfBytesRead is smaller than >>> nNumberOfBytesToRead (http://msdn.microsoft.com/en-us/library/ >>> aa365467%28VS.85%29.aspx). This didn't work because at the beginning >>> sometimes the Python script returns lpNumberOfBytesRead < >>> nNumberOfBytesToRead. It does some init and it takes time, I suppose. >>> 2. Use GetExitCodeProcess (http://msdn.microsoft.com/en-us/library/ >>> ms683189%28VS.85%29.aspx) on the child process handle. This method >>> didn't work either. For some reason the GetExitCodeProcess would >>> indicate that the process is still active. >>> Please help. >>> -Ilya. >> -- >> HLS > -- HLS
From: IlyaK on 2 Mar 2010 20:33
Thanks Hector. 1. get_cancel_text() is just an abstraction of a higher level function that checks if the user pressed some button. I know I don't block on it because2. I stepped through the code with the debugger. 2. If you know a better way to do this without piping, let me know. I don't need interactive IO. 3. I don't prematurely close the processor handle, if the hChildProcess != NULL. I do close the main thread handle. Do you think this is a problem? 4. Your code calls ReadFile until it returns false. I tried that, but I still get stuck. 5 Does it make any difference that I do all this in mfc application? Thank you, -Ilya. |