The code for IMAP, is a pain, the rewrite should clean all those

	hacks.  Something to remember for the mailbox2 is to do some
	smart caching, the current buffered stream is not good enough.

	* configure.in: VERSION is set to 0.0.9a for the alpha.
	* frm/frm.c: Protype for usage(). get_personal() should use
	const char *.
	* mail/mail.c (main): Use mailbox_scan() instead of
	mailbox_messages_count () it is faster for IMAP.

	* mailbox/mbx_imap.c(imap_get_message0): Remove the
	stream_setbufsiz() it is no longer used.
	(imap_message_readline imap_body_readline):
	Bug fix, readline will only consume up to '\n' even though the
	buffer is bigger.  So we have to adjust the offset/lines/size
	to reflect what we consume.  This is not very good for example
	doing:
	{
	   off_t offset = 14;
	   char buffer[512];
	   stream_readline (stream, buffer, sizeof buffer, offset, NULL);
	   ...
	}
	Will send to the impap server.
	C: g445 FETCH 4 BODY.PEEK[2]<0.2048>
        S: * 4 FETCH (BODY[2]<0> {739}
	S: g445 OK FETCH completed

	This is waste since the first line is only 37 bytes.  We will
	have to come up on the second API with some smart caching.

	* mailbox/memory_stream.c (_memory_write): Nasty bug
	was not updating the stream size.
	* mailbox/stream.c(stream_readline): Take care of known
	case where the buflen is 0 or 1.

 2001-10-14  Alain Magloire
 
-	The code for property.c has been change for a more simple approach,
-	it is the same approach as in mail/util.c(environment) settings.
-	The old code was simply overkill, property_t was seldomly use
-	if at all and no need for heavy hash table and all that jazz.
+	The code for IMAP, is a pain, the rewrite should clean all those
+	hacks.  Something to remember for the mailbox2 is to do some
+	smart caching, the current buffered stream is not good enough.
+
+	* configure.in: VERSION is set to 0.0.9a for the alpha.
+	* frm/frm.c: Protype for usage(). get_personal() should use
+	const char *.
+	* mail/mail.c (main): Use mailbox_scan() instead of
+	mailbox_messages_count () it is faster for IMAP.
+
+	* mailbox/mbx_imap.c(imap_get_message0): Remove the
+	stream_setbufsiz() it is no longer used.
+	(imap_message_readline imap_body_readline):
+	Bug fix, readline will only consume up to '\n' even though the
+	buffer is bigger.  So we have to adjust the offset/lines/size
+	to reflect what we consume.  This is not very good for example
+	doing:
+	{
+	   off_t offset = 14;
+	   char buffer[512];
+	   stream_readline (stream, buffer, sizeof buffer, offset, NULL);
+	   ...
+	}
+	Will send to the impap server.
+	C: g445 FETCH 4 BODY.PEEK[2]<0.2048>
+        S: * 4 FETCH (BODY[2]<0> {739}
+	S: g445 OK FETCH completed
+
+	This is waste since the first line is only 37 bytes.  We will
+	have to come up on the second API with some smart caching.
+
+	* mailbox/memory_stream.c (_memory_write): Nasty bug
+	was not updating the stream size.
+	* mailbox/stream.c(stream_readline): Take care of known
+	case where the buflen is 0 or 1.
 
-	* mailbox/property.c:  New code, the same as mail/util.c
-	environment.
+2001-10-14  Alain Magloire
+
+	The code for property.c has been changed for a more simple approach,
+	it is the same approach as mail/util.c(environment) settings.
+	The old code was simply overkill, property_t was seldomly used,
+	if at all, no need for heavy hash table and all that jazz.
+
+	* mailbox/property.c:  New code, the same as mail/util.c environment.
 	* mailbox/mbx_mbox.c (_mbx_init): Remove property_add_defaults() etc ..
 	no longer in use in the initialisation code.
 	* mailbox/mbx_imap.c: Likewised
 	* mailbox/mbx_pop.c: Likewised
 	* mailbox/mbx_mh.c: Likewised
-	* mailbox/header.c: Remove property code.
-	* mailbox/body.c: Remove property code.
-	* mailbox/message.c: Remove property code.
-	* mailbox/filter.c: Change the code according to the new property_t.
-	* mailbox/filter_rfc822.c: Change the code according to the new
-	property_t.
-	* include/mailutils/header.h: Remove property.
-	* include/mailutils/body.h: Remove property.
-	* include/mailutils/message.h: Remove property.
-	* include/mailutils/property.h: Remove property_add_defaults()
+	* mailbox/header.c: Removed property code.
+	* mailbox/body.c: Removed property code.
+	* mailbox/message.c: Removed property code.
+	* mailbox/filter.c: Changed the code according to the new property_t.
+	* mailbox/filter_rfc822.c: Changed the code according to the new
+	property_t structure.
+	* include/mailutils/header.h: Removed property.
+	* include/mailutils/body.h: Removed property.
+	* include/mailutils/message.h: Removed property.
+	* include/mailutils/property.h: Removed property_add_defaults()
 	property_set_valued () takes one more argument for overwrite.
-	* mailbox/include/header0.hy: Remove property field.
-	* mailbox/include/body0.hy: Remove property field.
-	* mailbox/include/message0.hy: Remove property field.
+	* mailbox/include/header0.hy: Removed property field.
+	* mailbox/include/body0.hy: Removed property field.
+	* mailbox/include/message0.hy: Removed property field.
 	* mailbox/mailbox.c(mailbox_get_property mailbox_set_property):
 	Changed Property code.
 	* mailbox/mailer.c(mailer_set_property mailer_get_property):
 	Changed Property code.
 
 	* mailbox/folder_imap.c (imap_search imap_expunge imap_status): Stubs
-	Search is not implemented yet, STATUS neither and EXPUNGE is not used.
+	SEARCH is not implemented yet, STATUS neither and EXPUNGE is not used.
 
-	* mailbox/list.c: Rename variable index to indx to shutup
+	* mailbox/list.c: Renamed variable index to indx to shutup
 	gcc whos confusing with index() when warning level was high.
 	* mailbox/md5-rsa.c: Likewised.
 
@@ -42,8 +79,8 @@
 	The debug object was being set on the folder not the mailbox.
 
 	* mailbox/mbx_imap.c (imap_messages_count): Attempt to reconnect
-	if the connection timeout.  Is this wise ?
-	(imap_scan0): Move gut of imap_scan() code tho here.
+	if the connection timeout.  Is this wise?
+	(imap_scan0): Moved gut of imap_scan() code here.
 	(imap_scan): Stub calling imap_scan0() with notification enable.
 	(imap_expunge): After CLOSE, call imap_scan0() wiht notification
 	disable.
@@ -52,13 +89,12 @@
 	of the recursive nature of the algorithm.  Sam should buy me
 	a beer for this.
 
-	* mailbox/parse822.c: Use more assert to catch errors.
+	* mailbox/parse822.c: Used more assert to catch errors.
 	(parse822_group): phrase was not free() in case of failure.
-	(parse822_mail_box): Dead if branch remove and free phrase.
+	(parse822_mail_box): Dead "else if" branch removed and free phrase.
 	(parse822_route): accumulator was not being freed.
-	(parse822_local_part): Move down st_free(more).
-	(parse822_domain): Move down st_free(more).
-
+	(parse822_local_part): Moved down st_free(more).
+	(parse822_domain): Moved down st_free(more).
 
 2001-10-14 Sergey Poznyakoff
 

 dnl Process this file with autoconf to procude a configure script.
 AC_INIT(mailbox/mailbox.c)
-AM_INIT_AUTOMAKE(mailutils, 0.0.9)
+AM_INIT_AUTOMAKE(mailutils, 0.0.9a)
 AM_CONFIG_HEADER(config.h)
 
 dnl Check for programs
@@ -247,9 +247,9 @@ fi
 
 dnl Output Makefiles
 AC_OUTPUT(Makefile mailutils.spec include/Makefile include/mailutils/Makefile
- m4/Makefile doc/Makefile doc/rfc/Makefile doc/texinfo/Makefile argp/Makefile 
- lib/Makefile lib/posix/Makefile mailbox/Makefile imap4d/Makefile 
- mailbox/include/Makefile from/Makefile mail/Makefile pop3d/Makefile 
- frm/Makefile sieve/Makefile messages/Makefile scripts/Makefile 
+ m4/Makefile doc/Makefile doc/rfc/Makefile doc/texinfo/Makefile argp/Makefile
+ lib/Makefile lib/posix/Makefile mailbox/Makefile imap4d/Makefile
+ mailbox/include/Makefile from/Makefile mail/Makefile pop3d/Makefile
+ frm/Makefile sieve/Makefile messages/Makefile scripts/Makefile
  libmu_scm/Makefile guimb/Makefile guimb/scm/Makefile
  MySql/Makefile)

@@ -33,6 +33,7 @@
 #include <mailutils/address.h>
 
 static int action (observer_t, size_t);
+static void usage (const char *argv);
 
 static struct option long_options[] =
 {
@@ -71,7 +72,7 @@ static int selected;
 
 /* Retrieve the Personal Name from the header To: or From:  */
 static int
-get_personal (header_t hdr, char *field, char *personal, size_t buflen)
+get_personal (header_t hdr, const char *field, char *personal, size_t buflen)
 {
   char hfield[512];
   int status;
@@ -193,7 +194,7 @@ action (observer_t o, size_t type)
   return 0;
 }
 
-void
+static void
 usage (const char *argv)
 {
   printf ("GNU Mailutils.\n");
@@ -323,7 +324,6 @@ main(int argc, char **argv)
     mailbox_t mbox;
     observer_t observer;
     observable_t observable;
-    int status;
 
     if ((status = mailbox_create_default (&mbox, mailbox_name) != 0)
 	|| (status = mailbox_open (mbox, MU_STREAM_READ) != 0))

@@ -359,7 +359,7 @@ main (int argc, char **argv)
 	  exit (EXIT_FAILURE);
 	}
 
-      if (mailbox_messages_count (mbox, &total) != 0)
+      if (mailbox_scan (mbox, 1, &total) != 0)
 	{
 	  util_error ("Can not read mailbox");
 	  exit (EXIT_FAILURE);

@@ -62,6 +62,8 @@ static int  imap_message_size     __P ((message_t, size_t *));
 static int  imap_message_lines    __P ((message_t, size_t *));
 static int  imap_message_fd       __P ((stream_t, int *));
 static int  imap_message_read     __P ((stream_t , char *, size_t, off_t, size_t *));
+static int  imap_message_readline __P ((stream_t, char *, size_t, off_t,
+					size_t *));
 static int  imap_message_uid      __P ((message_t, size_t *));
 
 /* mime_t API.  */
@@ -84,7 +86,10 @@ static int  imap_header_get_value __P ((header_t, const char*, char *, size_t, s
 static int  imap_header_get_fvalue __P ((header_t, const char*, char *, size_t, size_t *));
 
 /* body_t API.  */
-static int  imap_body_read        __P ((stream_t, char *, size_t, off_t, size_t *));
+static int  imap_body_read        __P ((stream_t, char *, size_t, off_t,
+					size_t *));
+static int  imap_body_readline    __P ((stream_t, char *, size_t, off_t,
+					size_t *));
 static int  imap_body_size        __P ((body_t, size_t *));
 static int  imap_body_lines       __P ((body_t, size_t *));
 static int  imap_body_fd          __P ((stream_t, int *));
@@ -372,9 +377,8 @@ imap_get_message0 (msg_imap_t msg_imap, message_t *pmsg)
         message_destroy (&msg, msg_imap);
         return status;
       }
-    /* We want the buffering.  */
-    stream_setbufsiz (stream, 128);
     stream_set_read (stream, imap_message_read, msg);
+    stream_set_readline (stream, imap_message_readline, msg);
     stream_set_fd (stream, imap_message_fd, msg);
     message_set_stream (msg, stream, msg_imap);
     message_set_size (msg, imap_message_size, msg_imap);
@@ -422,9 +426,8 @@ imap_get_message0 (msg_imap_t msg_imap, message_t *pmsg)
         message_destroy (&msg, msg_imap);
         return status;
       }
-    /* We want the buffering.  */
-    stream_setbufsiz (stream, 128);
     stream_set_read (stream, imap_body_read, body);
+    stream_set_readline (stream, imap_body_readline, body);
     stream_set_fd (stream, imap_body_fd, body);
     body_set_size (body, imap_body_size, msg);
     body_set_lines (body, imap_body_lines, msg);
@@ -947,6 +950,41 @@ imap_copy_message (mailbox_t mailbox, message_t msg)
 
 /* Message read overload  */
 static int
+imap_message_readline (stream_t stream, char *buffer, size_t buflen,
+		       off_t offset, size_t *plen)
+{
+  message_t msg = stream_get_owner (stream);
+  msg_imap_t msg_imap = message_get_owner (msg);
+  m_imap_t m_imap = msg_imap->m_imap;
+  size_t lines = msg_imap->message_lines;
+  int status;
+  size_t len = 0;
+  char *nl = buffer;
+
+  /* Start over.  */
+  if (offset == 0)
+    lines = 0;
+
+  buflen--; /* for the NULL. */
+  status = imap_message_read (stream, buffer, buflen, offset, &len);
+  if (len)
+    {
+      nl = memchr (buffer, '\n', len);
+      if (nl)
+	{
+	  nl++;
+	  *nl = '\0';
+	  msg_imap->message_lines = lines + 1;
+	}
+      else
+	nl = buffer + len;
+    }
+  if (plen)
+    *plen = nl - buffer;
+  return status;
+}
+
+static int
 imap_message_read (stream_t stream, char *buffer, size_t buflen,
 		   off_t offset, size_t *plen)
 {
@@ -954,6 +992,22 @@ imap_message_read (stream_t stream, char *buffer, size_t buflen,
   msg_imap_t msg_imap = message_get_owner (msg);
   m_imap_t m_imap = msg_imap->m_imap;
   f_imap_t f_imap = m_imap->f_imap;
+  char *oldbuf = NULL;
+  char newbuf[2];
+  int status;
+
+  /* This is so annoying, a buffer len of 1 is a killer. If you have for
+     example "\n" to retrieve from the server, IMAP will transform this to
+     "\r\n" and since you ask for only 1, the server will send '\r' only.
+     And ... '\r' will be stripped by (imap_readline()) the number of char
+     read will be 0 which means we're done .... sigh ...  So we guard by at
+     least ask for 2 chars.  */
+  if (buflen == 1)
+    {
+      oldbuf = buffer;
+      buffer = newbuf;
+      buflen = 2;
+    }
 
   /* Start over.  */
   if (offset == 0)
@@ -963,7 +1017,7 @@ imap_message_read (stream_t stream, char *buffer, size_t buflen,
   if (f_imap->state == IMAP_NO_STATE)
     {
       char *section = NULL;
-      int status = imap_messages_count (m_imap->mailbox, NULL);
+      status = imap_messages_count (m_imap->mailbox, NULL);
       if (status != 0)
 	return status;
 
@@ -983,7 +1037,11 @@ imap_message_read (stream_t stream, char *buffer, size_t buflen,
       MAILBOX_DEBUG0 (m_imap->mailbox, MU_DEBUG_PROT, f_imap->buffer);
       f_imap->state = IMAP_FETCH;
     }
-  return fetch_operation (f_imap, msg_imap, buffer, buflen, plen);
+  status = fetch_operation (f_imap, msg_imap, buffer, buflen, plen);
+
+  if (oldbuf)
+    oldbuf[0] = buffer[0];
+  return status;
 }
 
 static int
@@ -1585,6 +1643,22 @@ imap_header_read (header_t header, char *buffer, size_t buflen, off_t offset,
   msg_imap_t msg_imap = message_get_owner (msg);
   m_imap_t m_imap = msg_imap->m_imap;
   f_imap_t f_imap = m_imap->f_imap;
+  char *oldbuf = NULL;
+  char newbuf[2];
+  int status;
+
+  /* This is so annoying, a buffer len of 1 is a killer. If you have for
+     example "\n" to retrieve from the server, IMAP will transform this to
+     "\r\n" and since you ask for only 1, the server will send '\r' only.
+     And ... '\r' will be stripped by (imap_readline()) the number of char
+     read will be 0 which means we're done .... sigh ...  So we guard by at
+     least ask for 2 chars.  */
+  if (buflen == 1)
+    {
+      oldbuf = buffer;
+      buffer = newbuf;
+      buflen = 2;
+    }
 
   /* Start over.  */
   if (offset == 0)
@@ -1593,7 +1667,7 @@ imap_header_read (header_t header, char *buffer, size_t buflen, off_t offset,
   /* Select first.  */
   if (f_imap->state == IMAP_NO_STATE)
     {
-      int status = imap_messages_count (m_imap->mailbox, NULL);
+      status = imap_messages_count (m_imap->mailbox, NULL);
       if (status != 0)
         return status;
       /* We strip the \r, but the offset/size on the imap server is with that
@@ -1619,7 +1693,10 @@ imap_header_read (header_t header, char *buffer, size_t buflen, off_t offset,
       f_imap->state = IMAP_FETCH;
 
     }
-  return fetch_operation (f_imap, msg_imap, buffer, buflen, plen);
+  status = fetch_operation (f_imap, msg_imap, buffer, buflen, plen);
+  if (oldbuf)
+    oldbuf[0] = buffer[0];
+  return status;
 }
 
 /* Body.  */
@@ -1661,6 +1738,41 @@ imap_body_lines (body_t body, size_t *plines)
   return 0;
 }
 
+static int
+imap_body_readline (stream_t stream, char *buffer, size_t buflen, off_t offset,
+		    size_t *plen)
+{
+  message_t msg = stream_get_owner (stream);
+  msg_imap_t msg_imap = message_get_owner (msg);
+  m_imap_t m_imap = msg_imap->m_imap;
+  size_t lines = msg_imap->body_lines;
+  int status;
+  size_t len = 0;
+  char *nl = buffer;
+
+  /* Start over.  */
+  if (offset == 0)
+    lines = 0;
+
+  buflen--; /* for the NULL. */
+  status = imap_body_read (stream, buffer, buflen, offset, &len);
+  if (len)
+    {
+      nl = memchr (buffer, '\n', len);
+      if (nl)
+        {
+          nl++;
+          *nl = '\0';
+          msg_imap->body_lines = lines + 1;
+        }
+      else
+        nl = buffer + len;
+    }
+  if (plen)
+    *plen = nl - buffer;
+  return status;
+}
+
 /* FIXME: Send EISPIPE if trying to seek back.  */
 static int
 imap_body_read (stream_t stream, char *buffer, size_t buflen, off_t offset,
@@ -1673,7 +1785,7 @@ imap_body_read (stream_t stream, char *buffer, size_t buflen, off_t offset,
   f_imap_t f_imap = m_imap->f_imap;
   char *oldbuf = NULL;
   char newbuf[2];
-  int status = 0;
+  int status;
 
   /* This is so annoying, a buffer len of 1 is a killer. If you have for
      example "\n" to retrieve from the server, IMAP will transform this to

@@ -105,10 +105,10 @@ _memory_write (stream_t stream, const char *iptr, size_t isize,
       if (tmp == NULL)
 	return ENOMEM;
       mfs->ptr = tmp;
-      mfs->size = offset + isize;
       mfs->capacity = newsize;
     }
 
+  mfs->size = offset + isize;
   memcpy (mfs->ptr + offset, iptr, isize);
   if (nbytes)
     *nbytes = isize;

@@ -262,8 +262,14 @@ stream_readline (stream_t is, char *buf, size_t count,
 
   is->state = MU_STREAM_STATE_READ;
 
-  if (count == 0)
+  switch (count)
     {
+    case 1:
+      /* why would they do a thing like that?
+	 stream_readline() is __always null terminated.  */
+      if (buf)
+	*buf = '\0';
+    case 0: /* Buffer is empty noop.  */
       if (pnread)
 	*pnread = 0;
       return 0;