The code for IMAP, is a pain, the rewrite should clean all those

hacks. Something to remember for the mailbox2 is to do some smart caching, the current buffered stream is not good enough. * configure.in: VERSION is set to 0.0.9a for the alpha. * frm/frm.c: Protype for usage(). get_personal() should use const char *. * mail/mail.c (main): Use mailbox_scan() instead of mailbox_messages_count () it is faster for IMAP. * mailbox/mbx_imap.c(imap_get_message0): Remove the stream_setbufsiz() it is no longer used. (imap_message_readline imap_body_readline): Bug fix, readline will only consume up to '\n' even though the buffer is bigger. So we have to adjust the offset/lines/size to reflect what we consume. This is not very good for example doing: { off_t offset = 14; char buffer[512]; stream_readline (stream, buffer, sizeof buffer, offset, NULL); ... } Will send to the impap server. C: g445 FETCH 4 BODY.PEEK[2]<0.2048> S: * 4 FETCH (BODY[2]<0> {739} S: g445 OK FETCH completed This is waste since the first line is only 37 bytes. We will have to come up on the second API with some smart caching. * mailbox/memory_stream.c (_memory_write): Nasty bug was not updating the stream size. * mailbox/stream.c(stream_readline): Take care of known case where the buflen is 0 or 1.

The code for IMAP, is a pain, the rewrite should clean all those
hacks. Something to remember for the mailbox2 is to do some smart caching, the current buffered stream is not good enough. * configure.in: VERSION is set to 0.0.9a for the alpha. * frm/frm.c: Protype for usage(). get_personal() should use const char *. * mail/mail.c (main): Use mailbox_scan() instead of mailbox_messages_count () it is faster for IMAP. * mailbox/mbx_imap.c(imap_get_message0): Remove the stream_setbufsiz() it is no longer used. (imap_message_readline imap_body_readline): Bug fix, readline will only consume up to '\n' even though the buffer is bigger. So we have to adjust the offset/lines/size to reflect what we consume. This is not very good for example doing: { off_t offset = 14; char buffer[512]; stream_readline (stream, buffer, sizeof buffer, offset, NULL); ... } Will send to the impap server. C: g445 FETCH 4 BODY.PEEK[2]<0.2048> S: * 4 FETCH (BODY[2]<0> {739} S: g445 OK FETCH completed This is waste since the first line is only 37 bytes. We will have to come up on the second API with some smart caching. * mailbox/memory_stream.c (_memory_write): Nasty bug was not updating the stream size. * mailbox/stream.c(stream_readline): Take care of known case where the buflen is 0 or 1.
Alain Magloire
Commit 24361598 ... 24361598116c688aeabec17db0a133af03ed0d64 authored 2001-10-15 05:29:37 +0000 by Alain Magloire
Showing 7 changed files with 203 additions and 49 deletions
ChangeLog
configure.in
frm/frm.c
mail/mail.c
mailbox/mbx_imap.c
mailbox/memory_stream.c
mailbox/stream.c
--- a/ChangeLog
View file @2436159
+++ b/ChangeLog
View file @2436159
 2001-10-14  Alain Magloire
-	The code for property.c has been change for a more simple approach,
+	The code for IMAP, is a pain, the rewrite should clean all those
-	it is the same approach as in mail/util.c(environment) settings.
+	hacks.  Something to remember for the mailbox2 is to do some
-	The old code was simply overkill, property_t was seldomly use
+	smart caching, the current buffered stream is not good enough.
-	if at all and no need for heavy hash table and all that jazz.
+	* configure.in: VERSION is set to 0.0.9a for the alpha.
+	* frm/frm.c: Protype for usage(). get_personal() should use
+	const char *.
+	* mail/mail.c (main): Use mailbox_scan() instead of
+	mailbox_messages_count () it is faster for IMAP.
+	* mailbox/mbx_imap.c(imap_get_message0): Remove the
+	stream_setbufsiz() it is no longer used.
+	(imap_message_readline imap_body_readline):
+	Bug fix, readline will only consume up to '\n' even though the
+	buffer is bigger.  So we have to adjust the offset/lines/size
+	to reflect what we consume.  This is not very good for example
+	doing:
+	{
+	   off_t offset = 14;
+	   char buffer[512];
+	   stream_readline (stream, buffer, sizeof buffer, offset, NULL);
+	   ...
+	}
+	Will send to the impap server.
+	C: g445 FETCH 4 BODY.PEEK[2]<0.2048>
+        S: * 4 FETCH (BODY[2]<0> {739}
+	S: g445 OK FETCH completed
+	This is waste since the first line is only 37 bytes.  We will
+	have to come up on the second API with some smart caching.
+	* mailbox/memory_stream.c (_memory_write): Nasty bug
+	was not updating the stream size.
+	* mailbox/stream.c(stream_readline): Take care of known
+	case where the buflen is 0 or 1.
-	* mailbox/property.c:  New code, the same as mail/util.c
+2001-10-14  Alain Magloire
-	environment.
+	The code for property.c has been changed for a more simple approach,
+	it is the same approach as mail/util.c(environment) settings.
+	The old code was simply overkill, property_t was seldomly used,
+	if at all, no need for heavy hash table and all that jazz.
+	* mailbox/property.c:  New code, the same as mail/util.c environment.
 	* mailbox/mbx_mbox.c (_mbx_init): Remove property_add_defaults() etc ..
 	no longer in use in the initialisation code.
 	* mailbox/mbx_imap.c: Likewised
 	* mailbox/mbx_pop.c: Likewised
 	* mailbox/mbx_mh.c: Likewised
-	* mailbox/header.c: Remove property code.
+	* mailbox/header.c: Removed property code.
-	* mailbox/body.c: Remove property code.
+	* mailbox/body.c: Removed property code.
-	* mailbox/message.c: Remove property code.
+	* mailbox/message.c: Removed property code.
-	* mailbox/filter.c: Change the code according to the new property_t.
+	* mailbox/filter.c: Changed the code according to the new property_t.
-	* mailbox/filter_rfc822.c: Change the code according to the new
+	* mailbox/filter_rfc822.c: Changed the code according to the new
-	property_t.
+	property_t structure.
-	* include/mailutils/header.h: Remove property.
+	* include/mailutils/header.h: Removed property.
-	* include/mailutils/body.h: Remove property.
+	* include/mailutils/body.h: Removed property.
-	* include/mailutils/message.h: Remove property.
+	* include/mailutils/message.h: Removed property.
-	* include/mailutils/property.h: Remove property_add_defaults()
+	* include/mailutils/property.h: Removed property_add_defaults()
 	property_set_valued () takes one more argument for overwrite.
-	* mailbox/include/header0.hy: Remove property field.
+	* mailbox/include/header0.hy: Removed property field.
-	* mailbox/include/body0.hy: Remove property field.
+	* mailbox/include/body0.hy: Removed property field.
-	* mailbox/include/message0.hy: Remove property field.
+	* mailbox/include/message0.hy: Removed property field.
 	* mailbox/mailbox.c(mailbox_get_property mailbox_set_property):
 	Changed Property code.
 	* mailbox/mailer.c(mailer_set_property mailer_get_property):
 	Changed Property code.
 	* mailbox/folder_imap.c (imap_search imap_expunge imap_status): Stubs
-	Search is not implemented yet, STATUS neither and EXPUNGE is not used.
+	SEARCH is not implemented yet, STATUS neither and EXPUNGE is not used.
-	* mailbox/list.c: Rename variable index to indx to shutup
+	* mailbox/list.c: Renamed variable index to indx to shutup
 	gcc whos confusing with index() when warning level was high.
 	* mailbox/md5-rsa.c: Likewised.
@@ -42,8 +79,8 @@
 	The debug object was being set on the folder not the mailbox.
 	* mailbox/mbx_imap.c (imap_messages_count): Attempt to reconnect
-	if the connection timeout.  Is this wise ?
+	if the connection timeout.  Is this wise?
-	(imap_scan0): Move gut of imap_scan() code tho here.
+	(imap_scan0): Moved gut of imap_scan() code here.
 	(imap_scan): Stub calling imap_scan0() with notification enable.
 	(imap_expunge): After CLOSE, call imap_scan0() wiht notification
 	disable.
@@ -52,13 +89,12 @@
 	of the recursive nature of the algorithm.  Sam should buy me
 	a beer for this.
-	* mailbox/parse822.c: Use more assert to catch errors.
+	* mailbox/parse822.c: Used more assert to catch errors.
 	(parse822_group): phrase was not free() in case of failure.
-	(parse822_mail_box): Dead if branch remove and free phrase.
+	(parse822_mail_box): Dead "else if" branch removed and free phrase.
 	(parse822_route): accumulator was not being freed.
-	(parse822_local_part): Move down st_free(more).
+	(parse822_local_part): Moved down st_free(more).
-	(parse822_domain): Move down st_free(more).
+	(parse822_domain): Moved down st_free(more).
 2001-10-14 Sergey Poznyakoff
--- a/configure.in
View file @2436159
+++ b/configure.in
View file @2436159
 dnl Process this file with autoconf to procude a configure script.
 AC_INIT(mailbox/mailbox.c)
-AM_INIT_AUTOMAKE(mailutils, 0.0.9)
+AM_INIT_AUTOMAKE(mailutils, 0.0.9a)
 AM_CONFIG_HEADER(config.h)
 dnl Check for programs
@@ -247,9 +247,9 @@ fi
 dnl Output Makefiles
 AC_OUTPUT(Makefile mailutils.spec include/Makefile include/mailutils/Makefile
- m4/Makefile doc/Makefile doc/rfc/Makefile doc/texinfo/Makefile argp/Makefile 
+ m4/Makefile doc/Makefile doc/rfc/Makefile doc/texinfo/Makefile argp/Makefile
- lib/Makefile lib/posix/Makefile mailbox/Makefile imap4d/Makefile 
+ lib/Makefile lib/posix/Makefile mailbox/Makefile imap4d/Makefile
- mailbox/include/Makefile from/Makefile mail/Makefile pop3d/Makefile 
+ mailbox/include/Makefile from/Makefile mail/Makefile pop3d/Makefile
- frm/Makefile sieve/Makefile messages/Makefile scripts/Makefile 
+ frm/Makefile sieve/Makefile messages/Makefile scripts/Makefile
 libmu_scm/Makefile guimb/Makefile guimb/scm/Makefile
 MySql/Makefile)
--- a/frm/frm.c
View file @2436159
+++ b/frm/frm.c
View file @2436159
@@ -33,6 +33,7 @@
 #include <mailutils/address.h>
 static int action (observer_t, size_t);
+static void usage (const char *argv);
 static struct option long_options[] =
 {
@@ -71,7 +72,7 @@ static int selected;
 /* Retrieve the Personal Name from the header To: or From:  */
 static int
-get_personal (header_t hdr, char *field, char *personal, size_t buflen)
+get_personal (header_t hdr, const char *field, char *personal, size_t buflen)
 {
  char hfield[512];
  int status;
@@ -193,7 +194,7 @@ action (observer_t o, size_t type)
  return 0;
 }
-void
+static void
 usage (const char *argv)
 {
  printf ("GNU Mailutils.\n");
@@ -323,7 +324,6 @@ main(int argc, char **argv)
    mailbox_t mbox;
    observer_t observer;
    observable_t observable;
-    int status;
    if ((status = mailbox_create_default (&mbox, mailbox_name) != 0)
 	|| (status = mailbox_open (mbox, MU_STREAM_READ) != 0))
--- a/mail/mail.c
View file @2436159
+++ b/mail/mail.c
View file @2436159
@@ -359,7 +359,7 @@ main (int argc, char **argv)
 	  exit (EXIT_FAILURE);
 	}
-      if (mailbox_messages_count (mbox, &total) != 0)
+      if (mailbox_scan (mbox, 1, &total) != 0)
 	{
 	  util_error ("Can not read mailbox");
 	  exit (EXIT_FAILURE);
--- a/mailbox/mbx_imap.c
View file @2436159
+++ b/mailbox/mbx_imap.c
View file @2436159
@@ -62,6 +62,8 @@ static int  imap_message_size     __P ((message_t, size_t *));
 static int  imap_message_lines    __P ((message_t, size_t *));
 static int  imap_message_fd       __P ((stream_t, int *));
 static int  imap_message_read     __P ((stream_t , char *, size_t, off_t, size_t *));
+static int  imap_message_readline __P ((stream_t, char *, size_t, off_t,
+					size_t *));
 static int  imap_message_uid      __P ((message_t, size_t *));
 /* mime_t API.  */
@@ -84,7 +86,10 @@ static int  imap_header_get_value __P ((header_t, const char*, char *, size_t, s
 static int  imap_header_get_fvalue __P ((header_t, const char*, char *, size_t, size_t *));
 /* body_t API.  */
-static int  imap_body_read        __P ((stream_t, char *, size_t, off_t, size_t *));
+static int  imap_body_read        __P ((stream_t, char *, size_t, off_t,
+					size_t *));
+static int  imap_body_readline    __P ((stream_t, char *, size_t, off_t,
+					size_t *));
 static int  imap_body_size        __P ((body_t, size_t *));
 static int  imap_body_lines       __P ((body_t, size_t *));
 static int  imap_body_fd          __P ((stream_t, int *));
@@ -372,9 +377,8 @@ imap_get_message0 (msg_imap_t msg_imap, message_t *pmsg)
        message_destroy (&msg, msg_imap);
        return status;
      }
-    /* We want the buffering.  */
-    stream_setbufsiz (stream, 128);
    stream_set_read (stream, imap_message_read, msg);
+    stream_set_readline (stream, imap_message_readline, msg);
    stream_set_fd (stream, imap_message_fd, msg);
    message_set_stream (msg, stream, msg_imap);
    message_set_size (msg, imap_message_size, msg_imap);
@@ -422,9 +426,8 @@ imap_get_message0 (msg_imap_t msg_imap, message_t *pmsg)
        message_destroy (&msg, msg_imap);
        return status;
      }
-    /* We want the buffering.  */
-    stream_setbufsiz (stream, 128);
    stream_set_read (stream, imap_body_read, body);
+    stream_set_readline (stream, imap_body_readline, body);
    stream_set_fd (stream, imap_body_fd, body);
    body_set_size (body, imap_body_size, msg);
    body_set_lines (body, imap_body_lines, msg);
@@ -947,6 +950,41 @@ imap_copy_message (mailbox_t mailbox, message_t msg)
 /* Message read overload  */
 static int
+imap_message_readline (stream_t stream, char *buffer, size_t buflen,
+		       off_t offset, size_t *plen)
+{
+  message_t msg = stream_get_owner (stream);
+  msg_imap_t msg_imap = message_get_owner (msg);
+  m_imap_t m_imap = msg_imap->m_imap;
+  size_t lines = msg_imap->message_lines;
+  int status;
+  size_t len = 0;
+  char *nl = buffer;
+  /* Start over.  */
+  if (offset == 0)
+    lines = 0;
+  buflen--; /* for the NULL. */
+  status = imap_message_read (stream, buffer, buflen, offset, &len);
+  if (len)
+    {
+      nl = memchr (buffer, '\n', len);
+      if (nl)
+	{
+	  nl++;
+	  *nl = '\0';
+	  msg_imap->message_lines = lines + 1;
+	}
+      else
+	nl = buffer + len;
+    }
+  if (plen)
+    *plen = nl - buffer;
+  return status;
+}
+static int
 imap_message_read (stream_t stream, char *buffer, size_t buflen,
 		   off_t offset, size_t *plen)
 {
@@ -954,6 +992,22 @@ imap_message_read (stream_t stream, char *buffer, size_t buflen,
  msg_imap_t msg_imap = message_get_owner (msg);
  m_imap_t m_imap = msg_imap->m_imap;
  f_imap_t f_imap = m_imap->f_imap;
+  char *oldbuf = NULL;
+  char newbuf[2];
+  int status;
+  /* This is so annoying, a buffer len of 1 is a killer. If you have for
+     example "\n" to retrieve from the server, IMAP will transform this to
+     "\r\n" and since you ask for only 1, the server will send '\r' only.
+     And ... '\r' will be stripped by (imap_readline()) the number of char
+     read will be 0 which means we're done .... sigh ...  So we guard by at
+     least ask for 2 chars.  */
+  if (buflen == 1)
+    {
+      oldbuf = buffer;
+      buffer = newbuf;
+      buflen = 2;
+    }
  /* Start over.  */
  if (offset == 0)
@@ -963,7 +1017,7 @@ imap_message_read (stream_t stream, char *buffer, size_t buflen,
  if (f_imap->state == IMAP_NO_STATE)
    {
      char *section = NULL;
-      int status = imap_messages_count (m_imap->mailbox, NULL);
+      status = imap_messages_count (m_imap->mailbox, NULL);
      if (status != 0)
 	return status;
@@ -983,7 +1037,11 @@ imap_message_read (stream_t stream, char *buffer, size_t buflen,
      MAILBOX_DEBUG0 (m_imap->mailbox, MU_DEBUG_PROT, f_imap->buffer);
      f_imap->state = IMAP_FETCH;
    }
-  return fetch_operation (f_imap, msg_imap, buffer, buflen, plen);
+  status = fetch_operation (f_imap, msg_imap, buffer, buflen, plen);
+  if (oldbuf)
+    oldbuf[0] = buffer[0];
+  return status;
 }
 static int
@@ -1585,6 +1643,22 @@ imap_header_read (header_t header, char *buffer, size_t buflen, off_t offset,
  msg_imap_t msg_imap = message_get_owner (msg);
  m_imap_t m_imap = msg_imap->m_imap;
  f_imap_t f_imap = m_imap->f_imap;
+  char *oldbuf = NULL;
+  char newbuf[2];
+  int status;
+  /* This is so annoying, a buffer len of 1 is a killer. If you have for
+     example "\n" to retrieve from the server, IMAP will transform this to
+     "\r\n" and since you ask for only 1, the server will send '\r' only.
+     And ... '\r' will be stripped by (imap_readline()) the number of char
+     read will be 0 which means we're done .... sigh ...  So we guard by at
+     least ask for 2 chars.  */
+  if (buflen == 1)
+    {
+      oldbuf = buffer;
+      buffer = newbuf;
+      buflen = 2;
+    }
  /* Start over.  */
  if (offset == 0)
@@ -1593,7 +1667,7 @@ imap_header_read (header_t header, char *buffer, size_t buflen, off_t offset,
  /* Select first.  */
  if (f_imap->state == IMAP_NO_STATE)
    {
-      int status = imap_messages_count (m_imap->mailbox, NULL);
+      status = imap_messages_count (m_imap->mailbox, NULL);
      if (status != 0)
        return status;
      /* We strip the \r, but the offset/size on the imap server is with that
@@ -1619,7 +1693,10 @@ imap_header_read (header_t header, char *buffer, size_t buflen, off_t offset,
      f_imap->state = IMAP_FETCH;
    }
-  return fetch_operation (f_imap, msg_imap, buffer, buflen, plen);
+  status = fetch_operation (f_imap, msg_imap, buffer, buflen, plen);
+  if (oldbuf)
+    oldbuf[0] = buffer[0];
+  return status;
 }
 /* Body.  */
@@ -1661,6 +1738,41 @@ imap_body_lines (body_t body, size_t *plines)
  return 0;
 }
+static int
+imap_body_readline (stream_t stream, char *buffer, size_t buflen, off_t offset,
+		    size_t *plen)
+{
+  message_t msg = stream_get_owner (stream);
+  msg_imap_t msg_imap = message_get_owner (msg);
+  m_imap_t m_imap = msg_imap->m_imap;
+  size_t lines = msg_imap->body_lines;
+  int status;
+  size_t len = 0;
+  char *nl = buffer;
+  /* Start over.  */
+  if (offset == 0)
+    lines = 0;
+  buflen--; /* for the NULL. */
+  status = imap_body_read (stream, buffer, buflen, offset, &len);
+  if (len)
+    {
+      nl = memchr (buffer, '\n', len);
+      if (nl)
+        {
+          nl++;
+          *nl = '\0';
+          msg_imap->body_lines = lines + 1;
+        }
+      else
+        nl = buffer + len;
+    }
+  if (plen)
+    *plen = nl - buffer;
+  return status;
+}
 /* FIXME: Send EISPIPE if trying to seek back.  */
 static int
 imap_body_read (stream_t stream, char *buffer, size_t buflen, off_t offset,
@@ -1673,7 +1785,7 @@ imap_body_read (stream_t stream, char *buffer, size_t buflen, off_t offset,
  f_imap_t f_imap = m_imap->f_imap;
  char *oldbuf = NULL;
  char newbuf[2];
-  int status = 0;
+  int status;
  /* This is so annoying, a buffer len of 1 is a killer. If you have for
     example "\n" to retrieve from the server, IMAP will transform this to
--- a/mailbox/memory_stream.c
View file @2436159
+++ b/mailbox/memory_stream.c
View file @2436159
@@ -105,10 +105,10 @@ _memory_write (stream_t stream, const char *iptr, size_t isize,
      if (tmp == NULL)
 	return ENOMEM;
      mfs->ptr = tmp;
-      mfs->size = offset + isize;
      mfs->capacity = newsize;
    }
+  mfs->size = offset + isize;
  memcpy (mfs->ptr + offset, iptr, isize);
  if (nbytes)
    *nbytes = isize;
--- a/mailbox/stream.c
View file @2436159
+++ b/mailbox/stream.c
View file @2436159
@@ -262,8 +262,14 @@ stream_readline (stream_t is, char *buf, size_t count,
  is->state = MU_STREAM_STATE_READ;
-  if (count == 0)
+  switch (count)
    {
+    case 1:
+      /* why would they do a thing like that?
+	 stream_readline() is __always null terminated.  */
+      if (buf)
+	*buf = '\0';
+    case 0: /* Buffer is empty noop.  */
      if (pnread)
 	*pnread = 0;
      return 0;