LISTSERV mailing list manager LISTSERV 15.5

Help for XML-L Archives

XML-L Archives

XML-L Archives


Next Message | Previous Message
Next in Topic | Previous in Topic
Next by Same Author | Previous by Same Author
Chronologically | Most Recent First
Proportional Font | Monospaced Font


Join or Leave XML-L
Reply | Post New Message
Search Archives

Subject: Re: Using XML for log files
From: "Anders W. Tell" <[log in to unmask]>
Reply-To:[log in to unmask]
Date:Thu, 10 Sep 1998 20:10:12 +0200

text/plain (102 lines)

Peter Flynn wrote:

> > I'm exploring the use of XML as a logging facility (for audit,
> > error, warning, debug messages). Our organization and our
> > customers generate a large number of log files. These files
> > are passed around and processed in various ways by
> > computer programs. I would like to have all of them in a
> > common format so that we can make one viewer for them
> > and parsing is covered (to a large degree). I'm relatively
> > new to XML and I'm reading Mr. Goldfarbs book and a
> > couple of others and I have some initial questions.
> >
> > Is this a reasonable use of this technology?
> IMHO you'll get better performance from a non-XML format if this is raw
> data. Adding markup will increase the file size, possibly signficantly,
> and implies other restrictions (see below).

This is certainly true, especially for realtime or high volume
However compression techniques reduces the size.

An other problem is that logging take time (writing to disk etc.) and in
realtime logging
this is not acceptable. The solution is to batch the log entries and
write them in a custom
log format to persistent storage (db  files, etc.)

Have a  look at following pages where I have started to collect commonly
log processing activities.

This picture shows that a conversion step is necessary before a "custom"
log format  is usable by standardized log tools.

> It's certainly possible but I'd need to see some convincing arguments.
> The use of existing standard tools for parsing and display may just be
> enough to make it worthwhile.

This is exactly what XLF is trying to accomplish

> > Has anyone already done this and can I reuse their
> > efforts?
> My guess is that lots of people are investigating it. I don't know of
> any systems which currently log raw operational/performance data in an
> SGML-based format, although I'm sure there must be some around.

Have a look at the XLF initiative. <>

> > The log files are going to reside on Windows 95/NT
> > boxes. Are there physical limits to the file sizes based
> > on the current available parsing technology?
> Nothing in any parser I know limits your file size, although there are
> some theoretical limitations in the SGML Declaration for XML, and some
> operating system limitations on recursion and nesting. Plain file size
> is more likely to be limited by the extent of your disk and the
> stability of the DOS/Windows operating systems (size _does_ matter :-)
> > What is the current way that folks represent binary data
> > in XML? I'll have to put binary data into the log files.
> They don't, and you shouldn't. There are character sequences used in
> SGML-based systems which might possibly occur by accident in the wrong
> place in binary data, and get misrecognised as markup. Binary data is
> better stored in binary files than intermingled with ASCII or other
> printable-character-sequence data.

I agree with you that XML is best suited for text information but there
that allows you to add binary data to XML.

> My $0.02
> ///Peter

/  Financial Toolsmiths AB  /
/  Anders W. Tell           /

Back to: Top of Message | Previous Page | Main XML-L Page



CataList Email List Search Powered by the LISTSERV Email List Manager