John McEleney / mailutils

This speeds up common reading operations by factor of 4-6.

* include/mailutils/stream.h (mu_stream_readdelim): New proto.
* include/mailutils/sys/stream.h (_mu_stream) <readdelim>: New method.
* mailbox/stream.c (_stream_scandelim, _stream_readdelim): New functions.
(mu_stream_readdelim): New function.
(mu_stream_readline): Rewrite using mu_stream_readdelim.
(mu_stream_getdelim): Optimize.

* mailbox/amd.c (amd_body_stream_readdelim): New function.
(_amd_attach_message): Set the readdelim method.
* mailbox/header.c: Add a placeholder for readdelim method.
* mailbox/message.c (_message_stream_readdelim): New function.
(_message_stream_create): Set the readdelim method.
* mailbox/streamref.c (_streamref_readdelim): New function.
(mu_streamref_create_abridged): Set the readdelim method.

Sergey Poznyakoff authored 2010-09-08 16:06:03 +0300

* mailbox/amd.c (_amd_attach_message): Store body.
(amd_scan_message): Scan the message only if it has been modified
after the last scan.
Fix loop condition.
Store mhm->mtime on success.
(amd_message_stream_open): Enforce full buffering.
(amd_body_stream_read): Fix.
(amd_body_stream_seek): Don't call mu_stream_seek on mhm->stream.

Sergey Poznyakoff authored 2010-09-08 16:06:03 +0300

* mailbox/message.c (_check_stream_state): Bugfix.
(_message_stream_seek): Bugfix.
(_message_stream_read): Always try to fill out the buffer.
(_message_stream_create): Set MU_STREAM_SEEK flag bit.

Sergey Poznyakoff authored 2010-09-08 16:06:03 +0300

* mailbox/filter_iconv.c (mu_filter_iconv_create): Return
the created stream in *s.

Sergey Poznyakoff authored 2010-09-08 16:06:03 +0300

* include/mailutils/sys/stream.h (_mu_stream)<offset>: New member.
* mailbox/stream.c (_stream_fill_buffer): Propagate return value
from mu_stream_read_unbuffered.
(_stream_flush_buffer): Use stream->offset. Honor the `all' argument.
(mu_stream_seek): Rewrite using the `offset' member.
(mu_stream_read_unbuffered)
(mu_stream_write_unbuffered): Update offset.
(mu_stream_readline): Return EINVAL if size is 0.

Sergey Poznyakoff authored 2010-09-08 16:06:03 +0300

* comsat/action.c
* imap4d/fetch.c
* imap4d/search.c
* mh/burst.c
* mh/comp.c
* mh/forw.c
* mh/mh_ctx.c
* mh/mh_format.c
* mh/mh_list.c
* mh/mhn.c
* mh/pick.y
* pop3d/retr.c
* pop3d/top.c
* readmsg/msglist.c
* readmsg/readmsg.c

Sergey Poznyakoff authored 2010-09-08 16:06:03 +0300

* libmu_scm/mu_body.c (mu_scm_body_free): Unref the stream.
Return 0.
(mu-body-read-line, mu-body-write): Use stringrefs.
* libmu_scm/mu_mailbox.c (mu-mailbox-get-port): Use stringrefs.
* libmu_scm/mu_message.c (mu-message-copy): Use stringrefs.
Use mu_stream_copy instead of manually copying stream contents.
(mu-message-get-port): Use stringrefs.
* libmu_scm/mu_port.c (mu_port_free): Unref the stream.

Sergey Poznyakoff authored 2010-09-08 16:06:03 +0300

* examples/header.c: Use streamrefs.
* examples/mimetest.c: Likewise.
* examples/mta.c: Likewise.
* examples/nntpclient.c: Remove an unused variable.

Sergey Poznyakoff authored 2010-09-08 16:06:03 +0300

* mimeview/mimeview.c (mimeview_fp): Restore the variable.

Sergey Poznyakoff authored 2010-09-08 16:06:02 +0300

The `mail' and `frm' utilities pass tests.

* include/mailutils/stream.h (mu_stream_err): New proto.
* libproto/mbox/mbox.c (_msg_stream_setup)
(_msg_body_setup): Fix stream abridgement.
(mbox_envelope_date, mbox_envelope_sender): Fix seek position.
(uid_to_stream): Fix invocation of mu_stream_printf.
(append_message_to_stream): Likewise.
(mbox_expunge_unlocked): Rewind tempstr before copying its
contents back to the mailbox.
(mbox_expunge0): Open tempstr.
* libproto/mbox/mbox0.h (_mbox_message)<header_from>: Rename to
envel_from.
<header_from_end>: Rename to envel_from_end. All uses changed.
* libproto/mbox/mboxscan.c: Minor changes.

* mail/decode.c: Use streamrefs.
* mail/escape.c: Likewise.
* mail/msgset.y: Likewise.
* mail/pipe.c: Likewise.
* mail/print.c: Likewise.
* mail/top.c: Likewise.
* mail/write.c: Likewise.

* mail/send.c: Use streamrefs.
(mail_send0): Reset env->header after assigning it to
the message.

* mailbox/body.c (_body_seek): Fix a typo which produced
a recursive call.
* mailbox/file_stream.c (fd_truncate): New function.
(_mu_file_stream_create): Initialize str->stream.truncate.

* mailbox/header.c (header_parse): Fix parsing of blurbs lacking
terminating empty line.
(header_seek): Fix boundary checks.
Return new position in presult.
* mailbox/message.c (string_find_eoh): Rewrite to handle \n\n
split between two successive invocations.
(_header_fill): Update the invocation of string_find_eoh.

* mailbox/stream.c (mu_stream_destroy): Call mu_stream_close.
(mu_stream_get_flags, mu_stream_set_flags)
(mu_stream_clr_flags): Ignore internal
bits.
(mu_stream_err): New function.
(mu_stream_close): Close the stream only if it is not used by anyone else.
* mailbox/streamref.c (_streamref_read): Clear transport error if
ESPIPE is returned.
(_streamref_size): Take into account sp->end and sp->start.
(mu_streamref_create_abridged): Set pointer to 0.

Sergey Poznyakoff authored 2010-09-08 16:06:02 +0300

* mailbox/amd.c (amd_body_stream_seek): Fix boundary checking.
Return ESPIPE on boundary error.
* mailbox/header.c (header_seek): Likewise.
* mailbox/memory_stream.c (_memory_seek): Likewise.
* mailbox/message.c (_message_stream_seek): Likewise.
* mailbox/message_stream.c (_message_seek): Likewise.
* mailbox/stdio_stream.c (stdio_seek): Likewise.

* mailbox/file_stream.c (fd_open): Invert error condition.
Fix computation of open flags.
Return immediately if open returns -1.
(fd_seek): Fix boundary checking.
Return ESPIPE on boundary error.
* mailbox/mapfile_stream.c (_mapfile_open): Fix flags calculation.
(_mapfile_seek): Fix boundary checking.
Return ESPIPE on boundary error.

* mailbox/streamref.c (_streamref_read): Fix calculation of the
top size boundary.
Return 0 bytes if seek returns ESPIPE.
(_streamref_seek): Return ESPIPE on boundary error.

* mailbox/base64.c (_base64_decoder, _base64_encoder): Do
not dereference iobuf if cmd is mu_filter_init or mu_filter_done.
* mailbox/binflt.c (_copy_codec, _bit7_coder): Likewise.
* mailbox/crlfflt.c (_crlf_encoder, _crlf_decoder): Likewise.
* mailbox/linelenflt.c (_ll_encoder): Likewise.
* mailbox/qpflt.c (_qp_decoder, _qp_encoder): Likewise.

* mailbox/mailcap.c (mu_mailcap_parse): Fix loop condition.

Sergey Poznyakoff authored 2010-09-08 16:06:02 +0300

* examples/sfrom.c (main): Check return from mu_mailbox_messages_count.
* include/mailutils/sys/stream.h (_MU_STR_WRT): New constant.
* libproto/mbox/mbox.c (_msg_stream_setup): Fix 3rd argument to
mu_streamref_create_abridged.
* libproto/mbox/mboxscan.c (mbox_scan_internal): Don't use mailbox->stream
directly, because its offsets may get shifted by observers. Use streamref
instead.
* mailbox/argcv.c (argcv_get): Do nothing if argc <= 0.
* mailbox/mapfile_stream.c (_mapfile_seek): Fill the *presult.
* mailbox/memory_stream.c (_memory_done): Remove leftover free.
* mailbox/stream.c: Reorder functions.
(_mu_stream_create): Increase reference count (i.e. set it to 1)
before returning.
(mu_stream_seek): Call _stream_flush_buffer instead of
mu_stream_flush.
(mu_stream_read_unbuffered): Return 0 on EOF.
(mu_stream_write_unbuffered): Set _MU_STR_WRT bit.
(mu_stream_readline, mu_stream_getdelim): Check the actual
number of bytes read.
(mu_stream_flush): Call flush method only if _MU_STR_WRT is set.
Clear it.
* mailbox/streamref.c (_streamref_seek): Optimization for off==0 and
whence == MU_SEEK_CUR.
(_streamref_seek): Return new offset in ppos.
(mu_streamref_create_abridged): Set MU_STREAM_NO_CLOSE.
* mailbox/memory_stream.c (_memory_seek) : Return new offset in presult.
* mailbox/message_stream.c (_message_seek): Likewise.
* mailbox/stdio_stream.c (stdio_seek): Fix prototype.

Sergey Poznyakoff authored 2010-09-08 16:06:02 +0300

Does not compile yet.

New files:
* mailbox/streamref.c
* include/mailutils/sys/streamref.h
* include/mailutils/sys/file_stream.h
* include/mailutils/sys/header_stream.h
* include/mailutils/sys/mapfile_stream.h
* include/mailutils/sys/memory_stream.h
* include/mailutils/sys/message_stream.h
* include/mailutils/sys/prog_stream.h
* include/mailutils/sys/socket_stream.h
* include/mailutils/sys/stdio_stream.h
* include/mailutils/sys/stream.h
* include/mailutils/sys/streamtrans.h
* mailbox/prog_stream.c
* mailbox/stdio_stream.c
* mailbox/stream_printf.c
* mailbox/stream_vprintf.c
* mailbox/temp_file_stream.c
* mailbox/streamcpy.c
* mailbox/binflt.c
* mailbox/crlfflt.c
* mailbox/fltstream.c
* mailbox/qpflt.c
* mailbox/linelenflt.c

Removed files:
* mailbox/filter_rfc822.c
* mailbox/filter_trans.c

Moved files (with edits):
* libproto/include/filter0.h -> include/mailutils/sys/filter.h
* libproto/include/header0.h -> include/mailutils/sys/header.h
* libproto/include/message0.h -> include/mailutils/sys/message.h
* libproto/include/mime0.h -> include/mailutils/sys/mime.h

Modified files:
* comsat/action.c
* config/mailutils-config.c
* examples/base64.c
* examples/cpp/msg-send.cc
* examples/header.c
* examples/http.c
* examples/iconv.c
* examples/mimetest.c
* examples/msg-send.c
* examples/mta.c
* examples/murun.c
* examples/nntpclient.c
* examples/pop3client.c
* imap4d/append.c
* imap4d/auth_gsasl.c
* imap4d/fetch.c
* imap4d/preauth.c
* imap4d/search.c
* imap4d/util.c
* include/mailutils/body.h
* include/mailutils/folder.h
* include/mailutils/header.h
* include/mailutils/mailbox.h
* include/mailutils/mailer.h
* include/mailutils/mailutils.h
* include/mailutils/message.h
* include/mailutils/stream.h
* include/mailutils/sys/Makefile.am
* include/mailutils/sys/filter.h
* include/mailutils/sys/header.h
* include/mailutils/sys/message.h
* include/mailutils/sys/mime.h
* include/mailutils/sys/nntp.h
* include/mailutils/sys/pop3.h
* include/mailutils/sys/stream.h
* include/mailutils/types.hin
* lib/mailcap.c
* lib/mu_dbm.h
* libmu_argp/muinit.c
* libmu_auth/ldap.c
* libmu_auth/tls.c
* libmu_scm/Makefile.am
* libmu_scm/mu_body.c
* libmu_scm/mu_message.c
* libmu_scm/mu_port.c
* libmu_sieve/actions.c
* libmu_sieve/extensions/pipe.c
* libmu_sieve/extensions/spamd.c
* libmu_sieve/extensions/vacation.c
* libproto/mailer/smtp.c
* libproto/mbox/mboxscan.c
* libproto/mbox/mbox.c
* libproto/mbox/mbox0.h
* libproto/nntp/nntp_article.c
* libproto/nntp/nntp_carrier.c
* libproto/nntp/nntp_destroy.c
* libproto/nntp/nntp_ihave.c
* libproto/nntp/nntp_post.c
* libproto/nntp/nntp_readline.c
* libproto/nntp/nntp_sendline.c
* libproto/pop/pop3_carrier.c
* libproto/pop/pop3_destroy.c
* libproto/pop/pop3_readline.c
* libproto/pop/pop3_sendline.c
* libproto/pop/pop3_stat.c
* maidag/mailtmp.c
* mail/decode.c
* mail/escape.c
* mail/msgset.y
* mail/pipe.c
* mail/print.c
* mail/send.c
* mail/top.c
* mail/util.c
* mail/write.c
* mailbox/Makefile.am
* mailbox/amd.c
* mailbox/attachment.c
* mailbox/body.c
* mailbox/cfg_driver.c
* mailbox/cfg_format.c
* mailbox/debug.c
* mailbox/file_stream.c
* mailbox/folder.c
* mailbox/hdritr.c
* mailbox/header.c
* mailbox/mailbox.c
* mailbox/mailcap.c
* mailbox/mailer.c
* mailbox/mapfile_stream.c
* mailbox/memory_stream.c
* mailbox/message.c
* mailbox/message_stream.c
* mailbox/mime.c
* mailbox/mimehdr.c
* mailbox/mutil.c
* mailbox/prog_stream.c
* mailbox/progmailer.c
* mailbox/rfc2047.c
* mailbox/socket_stream.c
* mailbox/stream.c
* mailbox/stream_vprintf.c
* mailbox/tcp.c
* mailbox/vartab.c
* mailbox/wicket.c
* mh/burst.c
* mh/comp.c
* mh/compcommon.c
* mh/forw.c
* mh/mh_ctx.c
* mh/mh_format.c
* mh/mh_init.c
* mh/mh_list.c
* mh/mh_whatnow.c
* mh/mhl.c
* mh/mhn.c
* mh/pick.y
* mh/repl.c
* mimeview/mimeview.c
* mu-aux/generr.awk
* pop3d/extra.c
* pop3d/pop3d.h
* pop3d/retr.c
* pop3d/top.c
* readmsg/msglist.c
* readmsg/readmsg.c
* sieve/sieve.c

Sergey Poznyakoff authored 2010-09-08 16:06:02 +0300

* NEWS, configure.ac: Update.

Sergey Poznyakoff authored 2010-09-08 12:57:35 +0300

* imap4d/fetch.c (fetch_function_closure) <section_tag>: New member.
(_frt_body_text): Output ffc->section_tag after the secion part by
default.
(_frt_header, _frt_mime): Remove.
(_frt_header0): Rewrite as a general-purpose _frt_header function.
(parse_section_text): Set _frt_header as a driver function and
set section_tag accordingly, instead of selecting _frt_header/_frt_mime.
Set section_tag when selecting _frt_body_text.
(parse_section): Default to _frt_body_text and fall back to _frt_body
only when an empty section specifier is given.
* imap4d/testsuite/imap4d/fetch.exp: Update accordingly.

Sergey Poznyakoff authored 2010-09-07 13:39:07 +0300

* mailbox/acl.c (expand_arg): Fix a memory leak.

Sergey Poznyakoff authored 2010-08-26 12:42:04 +0300

* examples/mimetest.c (message_display_parts): Remove
not needed locals.

Sergey Poznyakoff authored 2010-08-12 17:09:47 +0300

* mailbox/mime.c (mu_mime_get_num_parts): Assume MIME message
is not scanned if nmtp_parts are 0 and boundary is NULL.
* mailbox/rfc2047.c (mu_rfc2047_decode): Free the buffer prior
to returning a non-zero status.
* mailbox/message.c (mu_message_destroy): Install a kludge
to work over the slopy ref semantics.
* mailbox/auth.c (mu_authority_destroy): Free auth_methods
list.
* mailbox/locker.c (destroy_dotlock): Free data.dot.nfslock.

Sergey Poznyakoff authored 2010-08-12 16:51:24 +0300

* examples/Makefile.am (mta_CPPFLAGS): New variable.

Sergey Poznyakoff authored 2010-06-02 22:08:18 +0300

* libmu_scm/mu_mailbox.c (mu-mailbox-get-size): New function.

Sergey Poznyakoff authored 2010-06-02 22:06:58 +0300

Delivery of a signal when a previous signal was being handled caused
mailbox corruption.

* imap4d/imap4d.c (imap4d_child_signal_setup): New function.
(imap4d_mainloop): Call imap4d_child_signal_setup.
* imap4d/signal.c (imap4d_child_signal): Reset all signal handlers
before doing actual job.
* lib/signal.c (mu_set_signals): Use sigaction.

Sergey Poznyakoff authored 2010-05-26 10:36:49 +0300

* movemail/movemail.c: New options: --ignore-errors and --program-id.
(_compare_msgno): Remove now unneded static.
(main): Handle --ignore-errors and --program-id settings.
Optimize the main loop for the case when --uidls is given.
Return 1 on errors.
* doc/texinfo/programs.texi: Document new features.

Sergey Poznyakoff authored 2010-05-20 16:29:36 +0300

* sql/mysql.c (mu_mysql_connect): Set CLIENT_MULTI_RESULTS flag.

Sergey Poznyakoff authored 2010-05-20 12:15:36 +0300

* mailbox/url.c (_url_path_default): Fix improper use of
sizeof (a typo, probably).

Sergey Poznyakoff authored 2010-05-18 14:05:36 +0300

* mh/mh_getopt.c (mh_opt_notimpl, mh_opt_notimpl_warning): New
functions.
* mh/mh_getopt.h (mh_opt_notimpl, mh_opt_notimpl_warning): New
protos.
* mh/ali.c (doc): Remove 'not implemented' stanza.
* mh/pick.c: Likewise.
* mh/anno.c: Use mh_opt_notimpl and mh_opt_notimpl_warning
for not implemented options.
* mh/comp.c: Likewise.
* mh/forw.c: Likewise.
* mh/mhn.c: Likewise.
* mh/refile.c: Likewise.
* mh/repl.c: Likewise.
* mh/scan.c: Likewise.
* mh/send.c: Likewise.

Sergey Poznyakoff authored 2010-05-13 16:54:03 +0300

* mh/mh_global.c (mh_read_profile): Set MU folder directory to
mh_get_dir.  This fixes the bug reported by Peter S Galbraith
(Id. <15312.1273667793@mixed>, ref.
http://sourceforge.net/mailarchive/forum.php?thread_name=15904.1273699954%40mixed&forum_name=mh-e-devel)

Sergey Poznyakoff authored 2010-05-13 16:21:53 +0300

Sergey Poznyakoff authored 2010-05-11 16:32:35 +0300

* gint: Update to 4e48677eee5a
* configure.ac (--without-guile): Remove option. Handled by GINT_INIT.
(GINT_INIT): Don't call it within a conditional.

Sergey Poznyakoff authored 2010-05-11 14:37:23 +0300

* mailbox/rfc2047.c (getword): Initialize *pret to NULL after freeing
it.

Sergey Poznyakoff authored 2010-05-07 00:15:56 +0300

* libproto/include/mime0.h (struct _mu_mime): Use size_t for
offsets, ssize_t for line_ndx.

Sergey Poznyakoff authored 2010-04-29 14:08:01 +0300

* mailbox/mimehdr.c (_header_get_param): Skip superfluous semicolons.

Sergey Poznyakoff authored 2010-04-29 13:59:30 +0300

* libproto/include/mime0.h (struct _mu_mime): New member line_size.
* mailbox/mime.c (_mime_setup_buffers): Use line_size instead of
MIME_MAX_HDR_LEN.
(_mime_parse_mpart_message): Reallocate cur_line as necessary.
(mu_mime_create): Initialize line_size.

Sergey Poznyakoff authored 2010-04-29 13:38:32 +0300

* mailbox/mimehdr.c (_header_get_param): Tolerate unquoted value parts
which contain tspecials, except ';'.

Sergey Poznyakoff authored 2010-04-29 13:34:30 +0300

* mailbox/mimehdr.c (disp_segment_len): New function.
(mu_mimehdr_get_disp, mu_mimehdr_aget_disp): Use disp_segment_len.
Unquote the returned string.
(_header_get_param): Allow for whitespace before '"'. Unquote the
string before comparison.

Sergey Poznyakoff authored 2010-04-26 23:10:33 +0300

* mailbox/cfg_lexer.l (mu_cfg_parse_file): Parse only
regular files.

Sergey Poznyakoff authored 2010-04-18 21:10:10 +0300

* include/mailutils/stream.h (mu_stream_clr_flags): New function.
* mailbox/stream.c (mu_stream_clr_flags): New function.
* mailbox/mapfile_stream.c (_mapfile_open): Avoid unnecessary
boolean operation.
* mailbox/mutil.c (mu_decode_filter): Use mu_stream_clr_flags
to clear bits.

Sergey Poznyakoff authored 2010-04-14 20:34:10 +0300

* mailbox/mimehdr.c (mu_mimehdr_get_disp)
(mu_mimehdr_aget_disp): Return disposition even if
no parameters have been given.

Sergey Poznyakoff authored 2010-04-14 20:19:10 +0300

* mailbox/filter.c (filter_destroy): Destroy the underlying
stream, unless flag is given.
(mu_filter_create): Treat last argument as stream flags.
* mailbox/filter_iconv.c (_icvt_destroy): Free the icvt_stream
structure.

* examples/mimetest.c (message_display_parts): Pass
MU_STREAM_NO_CLOSE to the invocation of mu_filter_create.
* mail/decode.c (display_submessage): Likewise.
* mailbox/attachment.c (mu_message_save_attachment): Likewise.
* mh/mh_list.c (eval_body): Likewise.
* mh/mhn.c (mhn_message_size, show_internal): Likewise.
(finish_text_msg): Likewise; call mu_stream_destroy

* imap4d/preauth.c (decode64_buf): remove unnecessary
call to mu_stream_destroy.
* libmu_auth/ldap.c (chk_md5, chk_smd5)
(chk_sha, chk_ssha): Likewise.

* mailbox/mutil.c: Fix indentation.

Sergey Poznyakoff authored 2010-04-14 18:23:42 +0300

* mailbox/rfc2047.c (getword): Change signature; return error
code.
(mu_rfc2047_decode): Reflect the above change.

Sergey Poznyakoff authored 2010-04-14 18:21:52 +0300

* gint: Upgrade.
* libmu_scm/Makefile.am: Initialize BUILT_SOURCES.
* libmu_scm/mu_address.c (address_get_fp): Change
signature to match those of mu_address_aget family.
(all functions): Use functions from mu_address_aget family.
Downcase argument names. Refer to them in the docstring
using @var notation.
* libmu_scm/mu_body.c: Downcase argument names. Refer to them
in the docstring using @var notation.
* libmu_scm/mu_logger.c: Likewise.
* libmu_scm/mu_mailbox.c: Likewise.
* libmu_scm/mu_message.c: Likewise.
* libmu_scm/mu_mime.c: Likewise.
* libmu_scm/mu_scm.c: Likewise.
* libmu_scm/mu_util.c: Likewise.

Sergey Poznyakoff authored 2010-04-14 11:38:41 +0300