PMPULL v0.11b ============= PMPULL.TXT - Copyright (c) 2021, Fred C. Macall Overview, System Requirements, and Limitations PMPULL and PMPULLR are designed to replace PMPOP's mail receiving or pulling function in a DOS Packet Driver based Pegasus Mail (PMAIL) installation. PMPULL (but not PMPULLR, presently) complements PMPOP's capabilities by working with POP3 Post Office servers that may offer or impose a requirement for secure (TLS/SSL) connections with their clients. PMPOP doesn't seem to support secure connections. Therefore, PMPULL should make Pegasus Mail useful with a group of POP3 servers that can't be accessed through PMPOP. PMPULLR is a "Real Mode" version of PMPULL, with lesser system requirements. Moreover, it lacks the OpenSSL support, for secure connections, that distinguishes PMPULL. Some, who (still) don't need to have a secure connection, may find it a happy alternative to PMPOP. Both PMPULL and PMPULLR are intended to pull mail from POP3 servers which support the "unique-id listing" (UIDL) command. UIDL is deemed an optional command in RFC 1939, the standard for such servers. PMPULL and, to a lesser extent, PMPULLR will be able to contact POP3 servers that don't support the UIDL command. But, they won't be able to pull any mail from such servers. As PMPULL and PMPULLR only receive or pull mail, PMSMTP32, PMSMTP, PMPOP, or a comparable substitute also will be required in one's installation, for sending mail. PMPULL is a companion or counterpart for PMSMTP32 and runs on IBM PC compatible systems, based on an 80386 or later processor, in DOS. Such systems also are expected to have 4 MB of RAM, or more, and a 32-bit DOS Protected Mode Interface (DPMI) service. The PMPUL11B.ZIP package includes CWSDPMI.EXE, which will provide the needed DPMI service in the absence of any existing one. I now run PMPULL, PMSMTP32, and Pegasus Mail v3.3, in DOS, on a Pentium-based PC I've named George. PMPULLR is a companion or counterpart for PMSMTP and should run successfully on any IBM PC compatible system that can run DOS, Pegasus Mail, and PMSMTP or PMPOP. Such systems include 8088-based PCs with DOS v3.3. However, I have not been able to give PMPULLR a complete test with a POP3 server. Because my ISP's mail service has become a "Google App", based on Gmail, which requires the secure connections that necessitate PMPULL. I hope to hear from everyone able to give PMPULLR a try in their environment. My PMPUL11B.ZIP package includes both PMPULL and PMPULLR and all of their supporting files. (Throughout this document, PMPOP is used as shorthand to refer to PDPMPOP.EXE, one of at least two versions of PMPOP. PDPMPOP.EXE is the version that is used in a DOS Packet Driver environment. During installation, it gets renamed PMPOP.EXE . Hence, my shorthand usage. In a similar fashion, I will tend to use PMPULL as shorthand for both it and PMPULLR. When I refer to PMPULLR, that will be to distinguish it from PMPULL.) More About Secure Connection Protocol(s) A problem with POP3 authentication, or login, is that performing it reveals the user's ID and password to anyone able to monitor the user's session. Open or public WiFi Access Points provide examples of arrangements that may allow many listeners to monitor a WiFi user's POP3 session. A protocol for using SSL encryption to thwart SMTP session monitoring was defined by Netscape, beginning in February 1995. A similar arrangement has become widely used for secure POP3 sessions, as well. The Netscape definition calls for starting an SSL setup procedure as soon as a TCP/IP session has been set up. As part of this startup, the client is expected to check or verify the server's security certificate. In order to recognize and abort a "man in the middle" intrusion, into the session. PMPULL does not perform this check. With the Netscape protocol, nothing related to the pending POP3 session gets exchanged (in the clear) before that SSL startup. The Netscape definition seems to have been successful. However, as a practical matter, it requires a designated or separate port on POP3 servers, for secure sessions. This has come to be port 995. RFC 2959 defines a somewhat different protocol for securing POP3 sessions. PMPULL has yet to support this arrangement. Once a secure session has been set up, a Netscape or legacy protocol using POP3 server sends the +OK response that corresponds to the beginning of a traditional POP3 session. Then, the protocol continues with a USER UserID command from the client. Of course, these things and everything that follows, in a secure session, will be encrypted. PMPULL supports a pop3sec= configuration option for determining whether the legacy security protocol is to be used, or not. As PMPULLR can't perform the security protocol, it ignores any pop3sec= configuration. If PMPULL's SSL setup procedure fails, it will indicate what happened and QUIT the session. So, configuring pop3sec=LegacySSL insures that PMPULL won't send one's UserID nor Password in the clear. How They Work Both PMPULL.EXE and PMPULLR.EXE start off by looking for their shared configuration file, PMPULL.CFG, in the directory from which they have been loaded. And, if necessary, in the "current DOS directory". Much of the configuration data is dedicated to the included WATTCP TCP/IP communications package. In particular, if PMPULL version 0.11b finds my_ip=BOOTP, my_ip=DHCP, or my_ip=EDHCP specified in the configuration file, it then attempts to obtain its TCP/IP configuration from a BOOTP or DHCP server. With its configuration complete, PMPULL checks its command line argument(s). If there aren't exactly two, it displays its Usage message and terminates. For the record, this message reads: Usage: PMPULL UserID Password PMPULL then attempts to contact the pop3host= specified mail server. By default, PMPULL will attempt to contact the specified mail server via its well known TCP/IP port 110. This port number may be changed to, say, 995 by appending :995 to the pop3host= specified mail server's name, in PMPULL.CFG. Once it has contacted the POP3 host, PMPULL will try to start SSL, if pop3sec=LegacySSL is configured. After a TLS/SSL session has been started successfully, or when pop3sec=None is configured, PMPULL will work just as PMPULLR does: First, they will look for the server's opening +OK response. Then, they will send a USER UserID command. And, look for another +OK response. Next, they will send a PASS Password command. And, look for another +OK response. All these things, other than Password, are displayed and may be logged with SCRIPT, as described below. PMPULL's next command is STAT. This requests the number of mail(s), held by the server for UserID, and their total size. If the response to STAT indicates at least one mail held, PMPULL proceeds with preparing to read its PMPULL.MRL (Mail Read List) control file from the maildir= configured directory. If PMPULL.MRL is found to be non-existent, PMPULL will prepare to write a new one. If PMPULL.MRL is found to exist, PMPULL will prepare to write an updated copy of it into PMPULL.MRU, in the maildir= configured directory. Each line, if any, in PMPULL.MRL describes a mail item that has been received. The layout of these lines is: lll mmddyy Xhhhhc.CNM snnn unique-idCL RF Where: lll is a leading zero padded three digit line length number which includes itself and the line's ending CR and LF characters. mmddyy is the date the described mail item was received. Xhhhhc.CNM is the name of the file that received the described mail item. In Xhhhhc.CNM, hhhh is a hash of the server's unique-id, for the described mail item. c provides for hash collision resolution. It often is 0 but may range up to 9 or A through F. When it first gets read in PMAIL, a mail item's name gets changed to !hhhhc.CNM . That is, the name's initial X gets changed to ! . This may depend on PMAIL's "General Preferences" entry "Leave read new mail new" being specified as Y . s is a status code letter which may be N or D. As each new line is added to what will become the updated .MRL file, it is given N(ew) status. In later run(s), attempt(s) may be made to DELE(te) the described mail item from the POP3 server. The line's status may get changed to D(ELEted) when such an attempt is made. These attempt(s) begin once a described mail item's age reaches the rettime= configuration value. If the mail item is found (still) held by the POP3 server. nnn is of variable length and may be as short as a single digit. It is the described mail item's number at the time it was pulled from the POP3 server. This number has little significance thereafter. unique-id is the server's unique-id for the described mail item. Perhaps needless to say, this value is expected to be persistent across sessions. These values enable PMPULL to keep track of what are old and new mail items available from the POP3 server. Next, PMPULL issues a UIDL command and reads the server's multi-line response or report into heap memory. Upon receiving a satisfactory response to the UIDL command, PMPULL will read PMPULL.MRL, if possible, and try to match each of its lines with an entry in the memorized UIDL report. All line(s) from PMPULL.MRL that match a UIDL report entry are checked for an age that has reached the rettime= configuration value, or more. All of these matching lines get copied to the PMPULL.MRU file being written. If they have reached the rettime= configured age, their copy is given D(ELEted) status. And, the matching UIDL report entry is marked for DELE(tion). Line(s) from PMPULL.MRL that don't match a UIDL report entry are checked for an age of twice the rettime= configuration value or at least 23 days. Lines that haven't reached that age also get copied to the PMPULL.MRU file being written. Older lines get discarded. Once the PMPULL.MRL file has been processed, any UIDL report entries that remain unmatched represent new mail items. PMPULL proceeds to issue a RETR nnn command for each of these. Each RETR(ieved) mail item is written into a file named Xhhhhc.CNM, such that ?hhhhc.CNM doesn't match the name of any existing file in the maildir= configured directory. After each mail item has been successfully RETR(ieved) and written, a new line describing it gets added to the PMPULL.MRL or PMPULL.MRU file being written. In the end, if nothing goes wrong, the new line(s) added to PMPULL.MRU will get added to PMPULL.MRL. Each line to be added to PMPULL.MRL also gets displayed on the console. While receiving the UIDL report and RETR(ieving) and writing each mail item, PMPULL watches for line(s) starting with period. If period is the only character on a line, it is taken as the end-of-message indication. Otherwise, if immediately followed by an additional period, a leading period is taken as a doubling period and removed from the written mail item. Also, while RETR(ieving) and writing each mail item, PMPULL may fold any long line(s) it contains. Two configuration items, textwid= and htmlwid= , are provided to control this possible folding. textwid= specifies the maximum length to be allowed for most mail lines. htmlwid= specifies the maximum length to be allowed for mail lines located within