Click to See Complete Forum and Search --> : know of a script to automatically parse emails?


gamblor01
10-17-2007, 12:24 PM
Does anyone out there know of a script that can automatically retrieve emails from an email address (via either POP3 or IMAP) and then automatically parse them? I'm basically trying to get emails automatically parsed, and then I'll call a script to automatically run and insert the parsed data into a MySQL database.

Alternatively, I found programs out there like JPEE. Does anyone have any experience with any free programs such as this? It seems like JPEE won't connect to an email server and parse the emails, but if I can get my email client to save them in a folder somewhere on disk, it looks like JPEE will automatically parse all files in that directory. I'm not 100% sure of that yet though.

Anyone have any experience with this?

ph34r
10-19-2007, 11:44 AM
Look for a mail archive program named mhonarc - there was discussion a few years ago when I was using it on how to get a message parsed by it and hten insterted into a database (instead of html file). I sorta did some of it by creating a php script with the mhonarc output (instead of html), but that was rather duct-tape-ish

gamblor01
11-30-2007, 02:46 AM
Ok so I'm almost there with my implementation. I got the google API key and put up a google maps on my website. I've also got it reading an XML file in the same directory which contains points to be plotted on the map. So basically at this point I just need to find some way to append a timestamp and the latitude/longitude to the XML file whenever a new email comes in. Basically:

- notice a new incoming email
- parse the email for the timestamp in the header as well as the gps coordinates in the message body
- append the tokens into the XML file with the proper tags

Instead of using Monharc, I found some PHP scripts out there that will parse incoming emails, but they of course require a mail server. Basically it looks like the mail server is supposed to feed the email to PHP as if it were coming from STDIN or something along those lines.

One of the examples I saw mentioned using exim. Can someone tell me how that works? I'm not so good on setting up email servers. Can I just use exim to retrieve email from a gmail account for example and redirect that to a PHP script? Or to use exim do I need to specifically own domain name or something? I'm not really sure how all of that works.

bwkaz
11-30-2007, 09:20 PM
If the PHP can parse an email, why not just feed it an email? The current PHP interface may require the message to be fed to it via stdin, but one stream is as good as any other -- you could probably rewrite the interface to use a different stream (that gets passed as a parameter). The other option is to see if PHP will let you close the current stdin and re-open it from a different file (e.g. do a redirection inside the PHP process: this is all that a shell redirection does); then you can call this function, then you can continue. :)

gamblor01
11-30-2007, 11:18 PM
Thanks bwkaz...but I think I have this working with fetchmail. Er well...I can at least get mail to go to my root@localhost but not any other users. I've got another thread working on that one. If I can just get the emails to arrive in the correct /var/spool/mail file then I'll be all set!!

Off to write my parser now...