Unoffical empeg BBS

Quick Links: Empeg FAQ | RioCar.Org | Hijack | BigDisk Builder | jEmplode | emphatic
Repairs: Repairs

Topic Options
#364956 - 06/10/2015 01:41 stdOut and stdErr handling ... can't find on StackOverflow...
tfabris
carpal tunnel

Registered: 20/12/1999
Posts: 31565
Loc: Seattle, WA
I need my C# program to interact with the stdOut and stdErr of a DOS console application. Reading its text output asynchronously and poking text back into it to answer its prompts.

It works, but I'm having a problem with it on one machine. Not certain why yet because I can't debug the machine that is having the problem. But I have a suspicion I'm wondering about. It worked fine until they fixed a problem with the machine being too slow. Now that the machine is fast enough, it fails.

I think it might be missing the very tip, the very start of the very first prompt of the app's output. Not certain, but I think that might be it.

Pseudo-code of what my program is doing:
Code:
           // Create a process object which will allow us to run the process and control it
            this.process = new Process
            {
                StartInfo = new ProcessStartInfo
                {
                    FileName = exeFileName,
                    WindowStyle = ProcessWindowStyle.Normal,
                    RedirectStandardOutput = true, // So that we can redirect and interpret the console output
                    RedirectStandardInput = true,
                    RedirectStandardError = true,
                    UseShellExecute = false,
                    ErrorDialog = false,
                }
            };

            // Special event handlers to allow the system to gather the text output from the running exe so that we can check its contents
            this.process.OutputDataReceived += new DataReceivedEventHandler(ConsoleOutputReceived);
            this.process.ErrorDataReceived += new DataReceivedEventHandler(ErrorOutputReceived);

            // Initialize the value of the text string that will be updated by the event handlers
            this.OutputText = string.Empty;

            // Launch the process and start its output event handlers
            this.process.Start();
			
			// ***************************** NOTE: CANNOT PLACE THESE NEXT TWO LINES BEFORE PROCESS.START.
			// ***************************** Runtime won't let you. Gets a runtime error that the process
			// ***************************** isn't started yet.
			// ***************************** I wonder if this is the problem: the process starts too quickly
			// ***************************** and the first burst of output is lost and never caught by
			// ***************************** these event handlers on that system?
            this.process.BeginOutputReadLine();
            this.process.BeginErrorReadLine();

			
			// ********** Then there is a loop here which waits for certain strings to be received and writes to the standard input
			// ********** The problem occurs because, on one system, no text ever appears. Well actually one blank line appears.
			// ********** Which is interesting because a blank line is the last line of the output from the program before it waits
			// ********** for user input.
			

      	// *********************************************************************
        // Farther down, these are the separate functions to handle the outputs:
        // *********************************************************************
			
        private void ConsoleOutputReceived(object sender, DataReceivedEventArgs e)
        {
            Console.WriteLine("StdOut: " + e.Data.SafeToStringNeutral());    // So I can see what's happening
            this.OutputText += e.Data.SafeToStringNeutral();
        }

        private void ErrorOutputReceived(object sender, DataReceivedEventArgs e)
        {
            Console.WriteLine("StdErr: " + e.Data.SafeToStringNeutral());    // So I can see what's happening
            this.OutputText += e.Data.SafeToStringNeutral();
        }



What I can't figure out is, in an architecture like the one above, how do I capture the output of those first few characters, the ones before it can get its redirection handlers running?

All the other options I'm seeing use process.StandardOutput.ReadToEnd() or process.StandardOutput.ReadLine(), but those sit there and hang until the process exits, and I can't have that because I need to interact with it.

I've searched the internet, but they always show variants of the Synchronous and Asynchronous examples cited here http://stackoverflow.com/questions/4291912/process-start-how-to-get-the-output ... I need a hybrid approach where I synchronously read the beginning lines without hanging, and then switch to asynchronous after those are all in.

Any ideas?
_________________________
Tony Fabris

Top
#364959 - 06/10/2015 08:25 Re: stdOut and stdErr handling ... can't find on StackOverflow... [Re: tfabris]
peter
carpal tunnel

Registered: 13/07/2000
Posts: 4172
Loc: Cambridge, England
This chap agrees with you: http://alabaxblog.info/2013/06/redirectstandardoutput-beginoutputreadline-pattern-broken/ sounds like a paper-bag bug in BeginOutputReadLine (as opposed to in the kernel). It's not totally clear to me how his Task Parallel-based solution makes the synchronous calls act like asynchronous ones, but it might work for you to put your "expect-like" behaviour in a version of his ReadStream callback.

Peter

Top
#364960 - 06/10/2015 10:24 Re: stdOut and stdErr handling ... can't find on StackOverflow... [Re: tfabris]
Roger
carpal tunnel

Registered: 18/01/2000
Posts: 5680
Loc: London, UK
Cunning plan: start the process suspended; then issue the BeginOutputReadLine; then resume the process...
_________________________
-- roger

Top
#364964 - 06/10/2015 15:45 Re: stdOut and stdErr handling ... can't find on StackOverflow... [Re: tfabris]
tfabris
carpal tunnel

Registered: 20/12/1999
Posts: 31565
Loc: Seattle, WA
Spectacular, you guys.

Peter, yes, that is exactly the thing I'm talking about. He seems to have a very detailed explanation for his solution not involving deadlocks. I'll see if I can do something similar in my code.

Roger,

That is also a great idea. Do you know how to start the process suspended in C#?
_________________________
Tony Fabris

Top
#364970 - 06/10/2015 17:57 Re: stdOut and stdErr handling ... can't find on StackOverflow... [Re: tfabris]
canuckInOR
carpal tunnel

Registered: 13/02/2002
Posts: 3212
Loc: Portland, OR
Originally Posted By: tfabris
Do you know how to start the process suspended in C#?

In C++, it's by providing the CREATE_SUSPENDED flag to CreateProcess(). Looks like C#'s Process() doesn't provide that same access, and you'll have to use PInvoke to call CreateProcess() yourself.

Top
#364972 - 06/10/2015 18:03 Re: stdOut and stdErr handling ... can't find on StackOverflow... [Re: canuckInOR]
tfabris
carpal tunnel

Registered: 20/12/1999
Posts: 31565
Loc: Seattle, WA
Originally Posted By: canuckInOR
In C++, it's by providing the CREATE_SUSPENDED flag to CreateProcess(). Looks like C#'s Process() doesn't provide that same access, and you'll have to use PInvoke to call CreateProcess() yourself.


Yep, I found that, too. That's a bit messier than I wanted it to be, but I understand it. If the other method doesn't work for me, then I'll try that.
_________________________
Tony Fabris

Top
#364977 - 06/10/2015 20:22 Re: stdOut and stdErr handling ... can't find on StackOverflow... [Re: peter]
tfabris
carpal tunnel

Registered: 20/12/1999
Posts: 31565
Loc: Seattle, WA
Originally Posted By: peter
This chap agrees with you: http://alabaxblog.info/2013/06/redirectstandardoutput-beginoutputreadline-pattern-broken/ sounds like a paper-bag bug in BeginOutputReadLine (as opposed to in the kernel). It's not totally clear to me how his Task Parallel-based solution makes the synchronous calls act like asynchronous ones, but it might work for you to put your "expect-like" behaviour in a version of his ReadStream callback.


A variant of this ended up being the correct solution. His code didn't work for me because it still got deadlocks where I didn't want them, so I had to modify it heavily. But basically I took his idea of using Task.Factory.StartNew to create a separate thread to use the synchronous Process.StandardOutput.Read() to do my own reader which did not lose any of the starting text from the program.

Thank you!
_________________________
Tony Fabris

Top
#364978 - 06/10/2015 20:27 Re: stdOut and stdErr handling ... can't find on StackOverflow... [Re: tfabris]
tfabris
carpal tunnel

Registered: 20/12/1999
Posts: 31565
Loc: Seattle, WA
My final code ended up being something like this.

Code:
            // Create a process object which will allow us to run the Process and control it for the duration of the test
            this.Process = new Process
            {
                StartInfo = new ProcessStartInfo
                {
                    FileName = ExeFile,
                    WindowStyle = ProcessWindowStyle.Normal,
                    RedirectStandardOutput = true, // So that we can redirect and interpret the console output
                    RedirectStandardInput = true,
                    RedirectStandardError = true,
                    UseShellExecute = false,
                    ErrorDialog = false,
                }
            };

            // Initialize the value of the text string that collects the 
            // console output which will be updated by the asynchronous tasks below
            this.OutputText = string.Empty;

            // Launch the process
            this.Process.Start();
            Console.WriteLine("Launched Process at: " + ExeFile);

            // Note: Cannot use this traditional method of asynchronous reads for this exe,
            // because it misses the first burst of output from the exe, causing test to fail:
            //     this.Process.OutputDataReceived += new DataReceivedEventHandler(ConsoleOutputReceived);
            // See this web page for details why:  http://alabaxblog.info/2013/06/redirectstandardoutput-beginoutputreadline-pattern-broken/

            // Can read from stderr text asynchronously though, since we are not waiting and depending upon its contents
            this.Process.ErrorDataReceived += new DataReceivedEventHandler(ErrorOutputReceived);
            this.Process.BeginErrorReadLine();
            Console.WriteLine("Console stderr output has been redirected.");

            // Create a cancelable, asynchronous task to read from the normal stdout stream using
            // the synchronous call, to work around the issue of missing the first burst of output
            // when using the traditional method of redirecting .OutputDataReceived
            var taskCanceler = new CancellationTokenSource();
            CancellationToken cancellationToken = taskCanceler.Token;
            Task outputReader = Task.Factory.StartNew
            (() =>
                {
                    // Keep track of the current line of output so that we can echo it to the console line-by-line
                    string currentLine = string.Empty;

                    // Infinitely loop while inside this special outputReader task
                    while (true)
                    {
                        // Drop out of the task if the task has been canceled
                        cancellationToken.ThrowIfCancellationRequested();

                        // Sleep a tiny amount while inside the loop, to prevent this task from churning the CPU
                        Thread.Sleep(1);

                        // Read from the stdout of the retdirected process output, in synchronous mode,
                        // to work around the issue where using the .OutputDataReceived feature misses
                        // the start of the output of fast-starting tasks; note that we cannot use
                        // "ReadToEnd()" or some such, because that would hang this thread, so we want
                        // to keep reading without stopping
                        int oneChar = this.Process.StandardOutput.Read();

                        // Process the character we read 
                        if (-1 != oneChar)
                        {
                            currentLine += string.Format("{0}", (char)oneChar);
                            this.OutputText += string.Format("{0}", (char)oneChar);
                            // If we get a linefeed, echo the buffered line to the console
                            if (10 == oneChar)
                            {
                                Console.Write("StdOut: " + currentLine);
                                currentLine = string.Empty;
                            }
                        }
                    }
                }
                , cancellationToken
            );
			
			
			// ************* Now down here you can do a nice leisurely loop where you look at the contents of 
			// this.OutputText and act upon it as needed.



            // Now we're done with the output reading thread, we can cancel it now
            taskCanceler.Cancel();


Thanks so much for your help!
_________________________
Tony Fabris

Top