Capturing Internet I/O

[Robert Collins]

Is there a way to capture input/output data streams during an Internet connection in an
assembly application?

Here is what I would like to do. My email client, which is MS Outlook Express connects to
Microsoft's POP3 server (pop3.email.msn.com on port 110) every 3 minutes to check for new mail.
I would like to see what Outlook Express is sending to the POP3 server and also see what the
POP3 server is responding back. I am trying to create my own POP3 client and I know the general
basic protocol but every time I try to connect to the server with my application I always get
an -ERR response from the server. Here is what I am doing (I am the client):

Code Select Expand


1) Client: Connect to Microsoft's POP3 server on port 110
2) Server: +OK BAY0-POP04 POP3 Server ready
3) Client: USER my_user_name
4) Server: -ERR command not implemented

Now, I have another POP3 server that I use also:

Code Select Expand


1) Client: Connect to xanthus.host4u.net
2) Server: +OK POP3 xanthus.host4u.net v2001.78rh server ready
3) Client: USER my_user_name
4) Server: +OK User name accepted, password please
5) Client: PASS my_password
6) Server: +OK Mailbox open, 5 messages

So, my POP3 program works for one server but not for the other. If I could write a program that
captures the i/o from Outlook to Microsoft's server I could then see what command strings are
being used. It can't be a mystery because other POP3 clients (other than Outlook) are able to
connect and get through.

[sheep]

Here's the pop3 RFC: http://www.faqs.org/rfcs/rfc1939.html

By the look of it, your client works fine but the server doesn't. What server are you talking about for the first thing? It's odd that a server would report that it doesn't support the USER command.

[pbrennick]

Robert,
Go to Japheth's website, http://www.japheth.de/, and download Joe.exe

This program is a mailreader (very nice, actually).  When you run this program, you will see a dialog window open with the mail server and it will show you all the commands and replies.

Paul

[Robert Collins]

By the look of it, your client works fine but the server doesn't. What server are you talking about for the first thing? It's odd that a server would report that it doesn't support the USER command.

In the sample that I posted it stated that it is 'pop3.email.msn.com' I know the server must work because Outlook Express uses that server to retrieve my email from the msn email data base. Also, if I use another email client (such as Endura or Opera) and I give them the name of the server and my user name + password they are able to retrieve my email.

Robert,
Go to Japheth's website, http://www.japheth.de/, and download Joe.exe

This program is a mailreader (very nice, actually).  When you run this program, you will see a dialog window open with the mail server and it will show you all the commands and replies.

Paul

Paul, I already had that application and it behaves exactly the same as the one I am trying to write. When I use Joe and connect to xanthus.host4u.net it works fine and is able to read my email from that server but when I connect to pop3.email.msn.com it does the same thing as my program; ie USER my-user_name ---> -ERR command not implemented.

The other two mail clients that I use (Endura and Opera) do not have a problem. This tells me that there must be some other protocol or something that is needed. This is why I need a port scanner to watch the input and output data for Outlook to and from the server so as to see what commands are being used.

[Ghirai]

You don't need a port scanner, you need a sniffer.

Go get Ethereal (free): http://ethereal.com, or direct download (for win): http://www.ethereal.com/distribution/win32/ethereal-setup-0.10.9.exe

[drhowarddrfine]

I remember reading about this some time ago that Microsofts email server doesn't work like every body elses does.  Not that that info is a lot of help but you might keep that in the back of your mind.

[Robert Collins]

I remember reading about this some time ago that Microsofts email server doesn't work like every body elses does.  Not that that info is a lot of help but you might keep that in the back of your mind.

Yeah, I already knew that. I just didnt know how different it is. It appears that other commerical mail client programs (like Endura and Opera, etc) obviously know what that difference is.

So far I have discovered this:

Microsoft's POP3 server doesn't accept the USER and PASS commands because they are encrypted...it goes something like this.........

Code Select Expand


1) Client: Connect to pop3.email.msn.com
2) Server: +OK BAY0-POP04 POP3 Server ready
3) Client: AUTH
4) Server: +OK
5) Client: AUTH MSN T1RMTVNTUAA.....................................
6) Server: +OK mailbox for user_name@msn.com has 3 messages(s)
7) CLient: LIST 1
etc, etc, etc


I was able to pick that out by using the sniffer that Ghirai pointed me to. I'm not sure what kind of encryption it is (maybe MD5) but that is what I have to now figure out. It also appears that this same method is used on their SMTP server. Before sending the MAIL TO command one has to go through this AUTH stuff.
Once authenticated then you can send the MAIL command and continue on as a normal SMTP conversation.

It pretty much boils down to this; if I can't figure it out then I am going to have to cancel my membership with MSN because I really need to write a mail client so as to retrieve my email and parse out unwanted crap that I get all the time.

[Robert Collins]

Here's the pop3 RFC: http://www.faqs.org/rfcs/rfc1939.html

Thanks, sheep. That link really comes in handy for things I need to know beyond my problem with MSN.

Robert,
Go to Japheth's website, http://www.japheth.de/, and download Joe.exe

Thanks, Paul. Although I already had that program I still appreciate your help (as always you seem to help me alot). It is a really cool program and it will come in handy later.

You don't need a port scanner, you need a sniffer.

Go get Ethereal (free): http://ethereal.com, or direct download (for win): http://www.ethereal.com/distribution/win32/ethereal-setup-0.10.9.exe

Thanks, Ghirai. That link was most useful. Now I just need to learn how to use it more efficiently. I can get some stuff but until I learn it, alot of the information seems to be missing on the packet feedbacks. But that's my problem. Thanks again.

[pbrennick]

Robert,
If you can read email from the mail client but you cannot write mail it means that server requires SMTP Authorization.  Some do and more and more are changing from the HELO protocol to the EHLO protocol.  That is probably your problem.  You will need the new commands and a username and password hashing algorythm.  Not entry level stuff!

Paul

[Relvinian]

Robert,

Here is a little trick to let you try sending/receiving command to a POP3/SMTP/NNTP server. You use telnet to communication.  ;-)

Example:
Open a CMD.EXE window (if using the NT kernel) or a COMMAND.COM (if using a 9x kernel).
type the follow:  telnet pop3.email.msn.com 110

Now you are connected.  Just start typing in commands (with any parameters necessary) and see what works and doesn't for that particular POP server.

According to the POP3 RFC -- the basic command ALL POP3 servers must support are:  USER, PASS and QUIT.

Relvinian

[Robert Collins]

Robert,
If you can read email from the mail client but you cannot write mail it means that server requires SMTP Authorization.  Some do and more and more are changing from the HELO protocol to the EHLO protocol.  That is probably your problem.  You will need the new commands and a username and password hashing algorythm.  Not entry level stuff!

Paul

Paul,

I can neither read from the MSN POP3 server nor write mail to the MSN SMTP server. Yes, When I try to do a SMTP I have to start out with EHLO instead of HELO. Then I will have to get through that AUTH stuff then I can do a MAIL TO etc etc. With the POP3, I have to issue a AUTH first, then MSN responds with +OK then I have to send AUTH MSN followed by that encrypted string. This I now know. So, I am now lwft with what you call the 'username and password hashing algorythm'. I'm not going to burn out over this matter but in time I will need to learn it so it's going to be a learning experience. Actually, I am only an entry level at assembly language programming so I fell that I have enough experience with the rest of the stuff that learning that encryption stuff won't be as difficult as it may sound it's just that I need to know where to start (my problems are usually 'where to start' rather that 'how to do' once I get a handle on it. 

[Robert Collins]

Robert,

Here is a little trick to let you try sending/receiving command to a POP3/SMTP/NNTP server. You use telnet to communication.  ;-)

Example:
Open a CMD.EXE window (if using the NT kernel) or a COMMAND.COM (if using a 9x kernel).
type the follow:  telnet pop3.email.msn.com 110

Now you are connected.  Just start typing in commands (with any parameters necessary) and see what works and doesn't for that particular POP server.

According to the POP3 RFC -- the basic command ALL POP3 servers must support are:  USER, PASS and QUIT.

Relvinian

Well, I can't argue with you about that but wheather MSN's POP3 server supports USER and PASS it doesn't appear that way in my case. I already know from experimenting around (thanks to the link from Ghirai) that I absolutely cannot talk with the POP3 server using the USER and PASS commands. As stated above I will always get a '-ERR command not implemented' response. But when I sent the server the AUTH command it allowed me to continue except that my AUTH code was incorrect. The packet sniffer proved to me that USER and PASS was not used by Outlook Express. Perhaps, the USER user_name and PASS password are all encrypted in the AUTH coded string, I don't know but it is possible.

[Relvinian]

Robert,

I haven't played with POP3 protocol since late 90's. It almost sounds like it is expecting a MD5 algo with the AUTH command you are trying to use. Check along that route and see if it helps you.

Relvinian

[pbrennick]

Relvinian,
Base64 is the type of encryption used.

Robert,
Until you negotiate a successful Authentication with the mail server, sending other commands are a total waste of time.  Fix your authentication first.

Paul

[Robert Collins]

Relvinian,
Base64 is the type of encryption used.

Robert,
Until you negotiate a successful Authentication with the mail server, sending other commands are a total waste of time.  Fix your authentication first.

Paul

Yes, Paul, I am quite aware of that. That is exactly what I am trying to do now. You say it is Base64. Are you 100% sure? I read it was MD5. I read an article on the APOP command and it stated that the encryption is MD5. I also read somewhere (can't remember where) that Microsoft uses the AUTH instead of the APOP command. Which ever it is I need to learn about this. Thanks.