IMAP4rev1 Client for Go

I have a scenario where a UID Search against Gmail might return a very large response. With the current implementation of transport.ReadLine(), the client errors and exits and I'm unable to get the search results.

To deal with longer lines without having to increase our buffer size, I've changed transport.ReadLine() to read the response in chunks until it encounters a non-bufio.ErrBufferFull error or an end of line.

I'm trying to get store of \Deleted flag working with Microsoft Exchange. I've adopted the code from demo1, I get back logging like this:

--- STORE --- imap: unexpected completion status ("PQZGK7 NO Command received in Invalid state.")

Logout reason: Microsoft Exchange Server 2010 IMAP4 server signing off. Close reason: end of stream Connection closing (flush=false)

I've tried browsing the source code and I'm not getting very far with that.

I have written a Go program that uses mxk/go-imap to copy an IMAP email account to another IMAP email account and to compare two IMAP accounts. It all works fine, including the copying of message flags and the comparing of mailbox flags.

However, I can't find anything in the mxk/go-imap interface that enables me to WRITE mailbox flags. I can read them by examining mailboxinfo.attrs which I can obtain by calling func (rsp *Response) MailboxInfo() *MailboxInfo but I can't see anything in the interface that allows me to write them. The method func (c *Client) Create(mbox string) (cmd *Command, err error) does not allow me to specify a set of flags, and there seems to be no additional method to allow me to do so.

I would really appreciate some help here because this is the last missing piece for an IMAP copy program that is otherwise copying and comparing everything else in an IMAP account perfectly. The mailbox flags is the only thing it doesn't copy correctly.

Ross Williams

Hi,

I've tried all day to delete messages. No matter what I try, it just doesn't work.

My code in its entirety that deals with looping through the mailbox.

I want to be able to select WHICH UIDs I delete. That's all.

Thanks

for cmd.InProgress() {
        // Wait for the next response (no timeout)
        c.Recv(-1)

        // Process command data
        for _, rsp = range cmd.Data {
            uid := imap.AsNumber(rsp.MessageInfo().UID)
            //msg_no := imap.AsNumber(rsp.MessageInfo().Seq)
            header := imap.AsBytes(rsp.MessageInfo().Attrs["RFC822.HEADER"])
            body := imap.AsBytes(rsp.MessageInfo().Attrs["RFC822.TEXT"])

            if msg, _ := mail.ReadMessage(bytes.NewReader(header)); msg != nil {
                xheaders := make(map[string]string)

                // Add X- Headers
                r := regexp.MustCompile("^X-(.*): (.*)")
                s_body := strings.Split(string(body), "\n")

                for _, value := range s_body {
                    res := r.FindAllString(value, -1)

                    if len(res) > 0 {
                        s_res := strings.Split(res[0], ": ")

                        h_key := s_res[0]
                        h_value := s_res[1]

                        // Fix up
                        h_key = strings.Replace(h_key, "[", "", -1)
                        h_value = strings.Replace(h_value, "\r", "", -1)

                        xheaders[h_key] = h_value
                    }
                }

                // Remove non Mailer-* headers
                for key, _ := range xheaders {
                    if !strings.Contains(key, "X-Mailer") {
                        delete(xheaders, key)
                    }
                }

                // Fix Date
                date := msg.Header.Get("Date")
                s_date := strings.Split(date, " (")

                mail := &Mail{
                    Uid:      uid,
                    Xheaders: xheaders,
                    From:     msg.Header.Get("FROM"),
                    Subject:  msg.Header.Get("Subject"),
                    Body:     string(body),
                    Time:     s_date[0],
                }

                res := process_message(mailbox_type, *mail)

                if res {
                    delset, _ := imap.NewSeqSet(string(uid))
                    c.UIDStore(delset, "+FLAGS.SILENT", imap.NewFlagSet(`\Deleted`))
                }

                fmt.Println(uid)
            }
        }

        c.Expunge(nil)

        cmd.Data = nil

        // Process unilateral server data
        /*for _, rsp = range c.Data {
            fmt.Println("Server data:", rsp)
        }*/
        c.Data = nil
    }

Hi,

While testing the client with different servers, I found that Yahoo IMAP is returning an unexpected trailing whitespace on the SEARCH results, e.g: "* SEARCH 61 62 63 " I'm far from an IMAP expert, so I wanted to ask: could this case be supported?

A test case to reproduce it:

--- a/imap/reader_test.go
+++ b/imap/reader_test.go
@@ -469,6 +469,9 @@ func TestReaderParse(t *testing.T) {
                {`* SEARCH 2 84 882`,
                        &Response{Tag: "*", Type: Data, Label: "SEARCH", Fields: []Field{"SEARCH", uint32(2), uint32(84), uint32(882)}}},

+               {`* SEARCH 2 84 882 `,
+                       &Response{Tag: "*", Type: Data, Label: "SEARCH", Fields: []Field{"SEARCH", uint32(2), uint32(84), uint32(882)}}},
+
                // Page 66
                {`* OK [ALERT] System shutdown in 10 minutes`,
                        &Response{Tag: "*", Type: Status, Status: OK, Info: "System shutdown in 10 minutes", Label: "ALERT", Fields: []Field{"ALERT"}}},

I still getting familiar with the code (reader.go) but will try to get a patch with a fix.

Currently in order to import this package one needs to do this:

import "github.com/mxk/go-imap/imap"

Imho this is a bit ugly and doesn't work well with the go get command as it expects the package to be in the root of the repo. I suggest renaming the package name to a nicer:

import "github.com/mxk/imap"

Dropping 'go-' from the name might be a bit confusing for people looking for imap libs from other languages, but I think that confusion will be quickly removed as soon as they see that it's written in go.

Andrew Gerrand had a similar suggestion in his talk here: http://talks.golang.org/2014/names.slide#17

I'm trying to get the library to connect to a Gmail server using some of the example code provided at http://godoc.org/github.com/mxk/go-imap/imap. I've stripped out all unnecessary code in order to just connect to a server. This is my example code:

package main

import "code.google.com/p/go-imap/go1/imap"
import "fmt"

func main() {
  client, _ := imap.Dial("imap.gmail.com")

  fmt.Println("Server says hello:", client.Data[0].Info)

  if client.State() == imap.Login {
    client.Login("<example e-mail>", "<example password>")
  }
}

For the sake of clarity and isolating the problem, I've removed the deferring of the logout and so on. Running the above code results in this error:

panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x21bc]

goroutine 16 [running]:
runtime.panic(0x22adc0, 0x3a24a4)
    /usr/local/Cellar/go/1.3.1/libexec/src/pkg/runtime/panic.c:279 +0xf5
main.main()
    /Users/kasper/go/src/test/imap.go:9 +0x1bc

goroutine 19 [finalizer wait]:
runtime.park(0x145d0, 0x3a5a60, 0x3a4569)
    /usr/local/Cellar/go/1.3.1/libexec/src/pkg/runtime/proc.c:1369 +0x89
runtime.parkunlock(0x3a5a60, 0x3a4569)
    /usr/local/Cellar/go/1.3.1/libexec/src/pkg/runtime/proc.c:1385 +0x3b
runfinq()
    /usr/local/Cellar/go/1.3.1/libexec/src/pkg/runtime/mgc0.c:2644 +0xcf
runtime.goexit()
    /usr/local/Cellar/go/1.3.1/libexec/src/pkg/runtime/proc.c:1445

goroutine 17 [syscall]:
runtime.goexit()
    /usr/local/Cellar/go/1.3.1/libexec/src/pkg/runtime/proc.c:1445
exit status 2

I've seen this problem referenced elsewhere, but I've been unable to find a solution. Any suggestions as to what might be wrong?

Hi,

I tried to fix the parsing of not quoted mailbox names containing squared brackets returned by a LIST/LSUB command (Dovecot has this behavior). RFC 3501 says that a mailbox name can be a quotes string, literal string or a astring:

mailbox-list = "(" [mbx-list-flags] ")" SP (DQUOTE QUOTED-CHAR DQUOTE / nil) SP mailbox

mailbox = "INBOX" / astring

astring = 1*ASTRING-CHAR / string

ASTRING-CHAR = ATOM-CHAR / resp-specials

ATOM-CHAR =

atom-specials = "(" / ")" / "{" / SP / CTL / list-wildcards / quoted-specials / resp-specials

resp-specials = "]"

The actual fix detects if the response is a LIST/LSUB response and then expects exactly 4 fields. The last one is the mailbox name. A new method parseAstring is added. The parseFields method is changed to accept a space as a stop character. A test is added.

Thanks!

I want to use Sourcegraph for go-imap code search, browsing, and usage examples. Can an admin enable Sourcegraph for this repository? Just go to https://sourcegraph.com/github.com/mxk/go-imap. (It should only take 30 seconds.)

Thank you!

This IMAP library fails for Microsoft Exchange servers because of the hardcoded UTF-8 charset:

imap: unexpected completion status (\"NHGVU5 NO [BADCHARSET (US-ASCII)] The specified charset is not supported.\")"

I've removed the default charset, and users may rather specify their own charset in the []imap.Field. e.g:

specs := []imap.Field{"CHARSET", "US-ASCII", "UNSEEN"}

I tested with a few different IMAP servers and all were happy with no charset set.

I'm having issues connecting to mail.privateemail.com, I'm using the same configuration that mbsync and others use to retrieve email from the server and mbsync works without issues.

The relevant code starts here, hosted on Gitlab

The configuration file looks like this:

{
  "host": "mail.privateemail.com",
  "port": 993,
  "tls": true,
  "tlsOptions": { "rejectUnauthorized": true },
  "username": "[email protected]",
  "password": "mysecretpassword",
  "onNewMail": "/home/jorge/.local/bin/getmail.sh",
  "onNewMailPost": "emacsclient  -e '(mu4e-update-index)'",
  "boxes": ["INBOX"],
  "__comment": "touch"
}

Please note: My code ignores tlsOptions at the moment.

This is the output my application shows:

[imap] 21:29:49 Connected to 198.54.122.60:993 (Tag=KFRUH)
[imap] 21:30:00 S:  (read tcp 192.168.0.10:59164->198.54.122.60:993: read: connection reset by peer)
[imap] 21:30:00 Close reason: protocol error
[imap] 21:30:00 Connection closing (flush=false)
[imap] 21:30:00 Greeting error: read tcp 192.168.0.10:59164->198.54.122.60:993: read: connection reset by peer
2017/08/22 21:30:00 [ERR] Cannot connect to mail.privateemail.com:993: read tcp 192.168.0.10:59164->198.54.122.60:993: read: connection reset by peer

This is the output if I use a non-encrypted port, basically it works fine:

[imap] 21:31:44 Connected to 198.54.122.60:143 (Tag=KTLIV)
[imap] 21:31:45 S: * OK [CAPABILITY STARTTLS IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN AUTH=LOGIN] Dovecot ready.
[imap] 21:31:45 Server greeting: Dovecot ready.
Server says hello: Dovecot ready.
[imap] 21:31:45 C: KTLIV1 STARTTLS
[imap] 21:31:45 S: KTLIV1 OK Begin TLS negotiation now
[imap] 21:31:45 TLS encryption enabled (cipher=0xC030)
[imap] 21:31:45 C: KTLIV2 CAPABILITY
[imap] 21:31:45 S: * CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN AUTH=LOGIN
[imap] 21:31:45 S: KTLIV2 OK Pre-login capabilities listed, post-login capabilities have more.
[imap] 21:31:45 C: KTLIV3 LOGIN "[email protected]" "mysecretpassword"
[imap] 21:31:46 S: * CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=O
RDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARC
HRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE NOTIFY METADATA QUOTA                                                                         
[imap] 21:31:46 S: KTLIV3 OK Logged in
[imap] 21:31:46 C: KTLIV4 CAPABILITY
[imap] 21:31:46 S: * CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=O
RDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARC
HRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE NOTIFY METADATA QUOTA                                                                         
[imap] 21:31:46 S: KTLIV4 OK Capability completed (0.001 + 0.000 secs).
[imap] 21:31:46 C: KTLIV5 EXAMINE "INBOX"
[imap] 21:31:46 S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
[imap] 21:31:46 S: * OK [PERMANENTFLAGS ()] Read-only mailbox.
[imap] 21:31:46 S: * 2 EXISTS
[imap] 21:31:46 S: * 0 RECENT
[imap] 21:31:46 S: * OK [UNSEEN 2] First unseen.
[imap] 21:31:46 S: * OK [UIDVALIDITY 1500785846] UIDs valid
[imap] 21:31:46 S: * OK [UIDNEXT 364] Predicted next UID
[imap] 21:31:46 S: * OK [HIGHESTMODSEQ 1026] Highest
[imap] 21:31:46 S: KTLIV5 OK [READ-ONLY] Examine completed (0.006 + 0.000 + 0.005 secs).
[imap] 21:31:46 C: KTLIV6 IDLE
[imap] 21:31:46 S: + idling

Not sure how to categorise this, but currently the IMAP client is spewing all kinds of warnings like such:

imap/client: 2016/10/21 17:31:52 response has not been handled: &{* OK HIGHESTMODSEQ [394056] Highest}

Maybe I'm wrong, but I don't seem to be able to specify custom handlers in the client; having such an option would allow me to catch expected "unhandled" messages while preserving the capability to warn if an unexpected unhandled response arises.

My server is running dovecot2 from Debian (jessie) by the way.

CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS LOGINDISABLED

I can't use Login function. And I don't understand how to use Authenticate

Thunderbird connects to server by STARTTLS using login and password w/o encryption.

Please, help me to connect to my imap server.