Capturing Internet I/O

[Relvinian]

Paul,

Like I mentioned in an earlier post in this thread, it has been since the late 90s since I played with POP3 protocol but I don't remember POP3 ever having Base64 for it Authincation method.  I know SMTP/NNTP do but not POP3.

On a site note, I looked at the latest RFC for the POP3 protocol and MD5 plus

Robert,
To check and see if a server supports APOP when you connect, the connect message sent from the POP3 server should specify in its initial mesage a "number" which represents a time. this is how you determine if the server supports APOP.  If not, it supports USER/PWRD combo.  If Microsoft indeed uses AUTH, I am ashamed at them.  But with the quick testing I did on their pop3.email.msn.com server, I do see they use the AUTH command.

Here is an example of what a server with APOP might send as their connect string:

Code Select Expand


S: +OK POP3 server ready <1896.697170952@dbc.mtview.ca.us>


So, here's how I would code it for now.  I would parse the initial connect string and look for some ID to see if it supports APOP. If so, use that as your authentication with MD5 algo.

If the APOP fails, I would try USER first before trying AUTH.

Relvinian


Relvinian

[Robert Collins]

Paul,

Like I mentioned in an earlier post in this thread, it has been since the late 90s since I played with POP3 protocol but I don't remember POP3 ever having Base64 for it Authincation method.  I know SMTP/NNTP do but not POP3.

On a site note, I looked at the latest RFC for the POP3 protocol and MD5 plus

Robert,
To check and see if a server supports APOP when you connect, the connect message sent from the POP3 server should specify in its initial mesage a "number" which represents a time. this is how you determine if the server supports APOP.  If not, it supports USER/PWRD combo.  If Microsoft indeed uses AUTH, I am ashamed at them.  But with the quick testing I did on their pop3.email.msn.com server, I do see they use the AUTH command.

Here is an example of what a server with APOP might send as their connect string:

Code Select Expand


S: +OK POP3 server ready <1896.697170952@dbc.mtview.ca.us>


So, here's how I would code it for now.  I would parse the initial connect string and look for some ID to see if it supports APOP. If so, use that as your authentication with MD5 algo.

If the APOP fails, I would try USER first before trying AUTH.
Relvinian


Relvinian

Relvinian,

OK, I think we are getting a little confused here about a few things. As stated above it will not accept any USER or PASS commands. Also, as you say above, But with the quick testing I did on their pop3.email.msn.com server, I do see they use the AUTH command then you already know that they use the AUTH and not the APOP command. From what i read somewhere else, Microsoft replaced the APOP command with the AUTH command. So, this will show that when connected you will not see +OK POP3 server ready <1896.697170952@dbc.mtview.ca.us> because that is only for APOP. I have no doubt that I must figure out how to do the encrytpion using MD5 (not Base64). I have a program that will convert a string to a MD5 encrypted string, but that is not my problem. What I don't know is what string I have to convert. This is very confusing to me because everytime Outlook connects to the server it passes a different coded string back to the server. Since it changes on each connection and there is no time-stamp from MSN and MSN accepts this encrypted string then there must be something else that this change is based on. But what? This is where my problem lies. If you look at the below snippit.........

Code Select Expand


1) Client: Connect to pop3.email.msn.com
2) Server: +OK BAY0-POP04 POP3 Server ready
3) Client: AUTH
4) Server: +OK
5) Client: AUTH MSN T1RMTVNTUAA.....................................
6) Server: +OK mailbox for user_name@msn.com has 3 messages(s)
7) CLient: LIST 1
etc, etc, etc


.......this is the only chain of commands that you can send to the server. The above was clipped from a packet that I was watching from a packet sniffer program. Any other series of  commands other than what you see in the above snippit will not work. The MD5 encryption is the string T1RMTVNTUAA...
but I have no idea how Outlook derives at this encryption. As you can see, upon connection, MSN does not respond with any time-stamp to be used as the basis of the MD5 encryption algo. So, here is where I am stuck. 

[Relvinian]

Robert,

The sequence of commands I gave to check progammically:

1) first check for the existance of APOP support
2) if #1 failes, try USER
3) if #2 fails, then resort to AUTH

Are so you could write a "generic" POP3 client.  If you are writing for a very specific POP3 mail server, I made the mistake in that. Then you would just use the AUTH command.

As for the MD5. The reason this changes with every single time you connect is because it also includes the Date/Time the string was MD5ed.  Find a good document explaining MD5 or a library of routines.  I'll dig around my past stuff on protocols and see if I can find one you could use to create your string.

Basically, what is contained in the encryption is your username, password, date, time and possibly something else. Also, one other possibility is that with the MSN addition to the AUTH command, it may be encrytping a .NET passport.

Some food for thought.


Relvinian

[pbrennick]

Relvinian,
Google it, EHLO is superceding HELO slowly but surely as an SMTP authentication, not POP3, if I confused the issue, I apologize.  But times, they are a changing and you may need to catch up.

Paul

[Relvinian]

Paul,

Robert is looking for information on POP3 (receiving emails) protocol not SMTP/NTTP (sending emails/news messages).

Relvinian

PS - Then again, I may be way off base!.   :P

[Relvinian]

Relvinian,
Google it, EHLO is superceding HELO slowly but surely as an SMTP authentication, not POP3, if I confused the issue, I apologize.  But times, they are a changing and you may need to catch up.

Paul

Yep, that's what I have found out with my SMTP clients I have written and are being used in commercial applications. You can get an overview of them from my Web site at http://www.sm-technologies.com/mat/mat.htm

I first check to see if the SMTP server supports EHLO and if it doesn't, I fall back to HELO. Then I find out what the best method for authentication is and use it that to authenticate them to the server (assuming you need it for that connection). I also support SSL and TLS secure connections with my software.

Relvinian

[pbrennick]

Relvinian,
I have been trying to to an EHLO connection via an assembly program and have failed miserably.  Can you give me any help?  You seem to have worked it out.  I am sorry I confused the issue earlier.
Paul

Also, Robert will eventually need that help, also.

[Robert Collins]

Paul,

Robert is looking for information on POP3 (receiving emails) protocol not SMTP/NTTP (sending emails/news messages).

Relvinian

PS - Then again, I may be way off base!.   :P

Yes, at the present time, I am trying to create my own POP3 client program. I don't think this program will ever be a generic client because when dealing with POP3 you really need to connect to your own EMAIL server to retrieve your own email. Currently I use Outlook Express to connect to my MSN POP3 account and I really don't have an issue with it but I would like to be able to do it myself so I can have absolute control over the incoming emails.

This is my first concern, ie POP3, then later I will try to implement a SMTP client as well (which in my case is also MSN and it too requires AUTH).

At present, I have been doing some MD5 experimenting. As far as MD5 goes, to me it is not such a big deal as how it works or how to translate a given string into a MD5 encrypted string but more so exactly what is it that Outlook Express uses as a string to translate into MD5. Like Relvinian said above, it does include my user name and password along with some other info. However, this presents a problem with me because I have no idea in exactly what order this information is constructed before it is converted to MD5. Also as I indicated earlier this MD5 string is different each time Outlook connects to the POP3 server. To make it different then something that has a changing value must be incorporated along with the user name and password. Now, Relvinian stated earlier that, based on the APOP method, the changing value is a time-stamp (Date and time of day). I can agree with this except in my particular case since MSN does not send a time-stamp to be used. In order to use a time-stamp the server must be made aware of this time-stamp. This means that Outlook cannot take the current time and use it because MSN's POP3 server would have no knowledge of it. So, in order for any MD5 to work as the output of Outlook Express it must be something that both parties have already agreed upon and further more it is probably something that the server changes each time a connection is made and passes that changed value to Outlook Express. This I have no idea since upon observing several sessions between Outlook Express and MSN's POP3 server it is always the same response from the server upon connecting. 

[Robert Collins]

Well now I'm really confused :(

Everything that I have been reading is that the encrypted string used with both the APOP and AUTH commands are MD5. So, I went and got a program that converts an ASCII string into MD5 encryption. The MD5 output of this program is ALWAYS a hexidecimal string.

For example:  "Hello, how are you?" would be "C5C8FB4DDE9EF50D4D258C6596428319"

But now either the program is incorrect or AUTH does not use MD5. Look at the sample POP3 session below...........

Code Select Expand


POP3 Client: Connecting to pop3.email.msn.com
POP3 Server: +OK BAY0-POP06 POP3 Server ready

POP3 Client: AUTH
POP3 Server: +OK
POP3 Server: MSN
POP3 Server: .

POP3 Client: AUTH MSN
POP3 Server: +

POP3 Client: AUTH MSN TlRMTVNTUA............UBQaX3vwAAQAA=
POP3 Server: + TlRMTVNTUA................AAr0zxJYLqxVNCQVkwLVBPUDA2

POP3 Client: TlRMTVNTUAADAAAAGAAY..................D6gBsAAReZ/MZhEJBWTAtUE9QMDZDYXBlcl9Cb2I=         
POP3 Server: +OK mailbox for user_name@msn.com  has 0 message(s)

POP3 Client: STAT
POP3 Server: +OK 0 0

POP3 Client: QUIT
POP3 Server: +OK mailbox for user_name@msn.com unchanged, BAY0-POP06 POP3 Server signing off 


...........note that the encrypted strings contains characters that are not hexidecimal. So, maybe it is BASE64. Not sure now.

[Robert Collins]

OK, it is not BASE64. I'm going to have to get another MD5 converter and compare the two programs' outputs.

[Tedd]

The data itself is base64 encdoed.
However, it's not encoded from plain text - ie. decoding the base64 will not give you the password (where's the security in that?)
The password is encoded using "MSN Authentication" and then that is what's encoded in base64. The reason for the base64 is to send binary data as plain text without it being mis-interpreted.
As for details on MSN Authentication - good luck! although I think there's a dll that provides it (otherwise where would it come from in the first place) so you could use the functions in that.

[Robert Collins]

Tedd,

You're right. After I posted the above I did some more research on the matter and was able to come up with the same conclusions. I'm sure there is somewhere a DLL that is used for this. Other professional mail client applications are able to do the same thing as Outlook so they must have access to not only 'MSN Authentication' but others as well.

[Tedd]

I knew I'd seen it somewhere :toothy

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/wininet/wininet/handling_authentication.asp
(MSN Auth provided by Msnsspc.dll)

So it would seem you can get access to it through the wininet dll :U

[Robert Collins]

I knew I'd seen it somewhere :toothy

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/wininet/wininet/handling_authentication.asp
(MSN Auth provided by Msnsspc.dll)

So it would seem you can get access to it through the wininet dll :U

Thanks Tedd. I will look into that option. However, for the time being I got around the problem by making a program that acts as a 'proxy' (I believe that is what it is called, correct me if I am wrong). My program simply sits in-between Outlook Express and my MSN POP3 mail server. The program just passes on to each party the input from both sides. It does what I want for now. I can now parse my incoming email and only pass back to Outlook what I want to keep and toss the rest of the crap. But I'm still working on building my own POP3 client but at least I can now take my time. 

[Robert Collins]

I knew I'd seen it somewhere :toothy

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/wininet/wininet/handling_authentication.asp

Ummmmm.....OK, I went to that link. It seems that I was there once before and had forgotton about it. As I read the info it appears to me that it is for HTTP and not POP3 protocols.