The future for HTML

XML and XLS

In case you weren't aware of it, the world of HTML as a language for creating web pages is changing. Soon the day will come when the latest internet browsers and web sites will use a new standard language known as XML and XSL.

Here is a brief introduction to help you prepare yourself for the new technology.

What is XML?

XML stands for Extensible Markup Language. It is said to be the universal standard means for describing (i.e. defining, storing and organising) data so that people and the software applications running on a computer, mobile phone and other devices can understand and use it to perform all sorts of tasks.

How does XML compare with HTML?

XML is not dissimilar to HTML in the sense that elements (i.e. tags or labels) are used around pieces of information to help define something. For example, in HTML we can write:

<h1>

  <center>Hello there</center>

  <i>by Joe Blow</i>

</h1>

In XML, we can write:

<book>

  <isbn>0 646 24066 8</isbn>

  <title>The Age of Light and Love: The Quest for Unity</title>

  <author>CJT</author>

  <publisher>SUNRISE</publisher>

  <publication-date>1995</publication-date>

</book>

The difference, however, lies in the elements themselves ie. the tags or labels. In HTML, the elements are pre-defined and cannot be changed. For instance you can't decide to use <bold></bold> to replace <b></b>. You can only use <b></b>. In XML, you create the elements how you like them. Hence you can write the above XML. But if you want, you could also write:

<book>

  <isbn>0 646 24066 8</isbn>

  <publication-title>The Age of Light and Love: The Quest for Unity</publication-title>

  <author-name>CJT</author-name>

  <publisher-name>SUNRISE</publisher-name>

  <publication-date>1995</publication-date>

</book>

XML has this level of flexibility.

To put it in a nutshell, HTML uses internationally-recognised labels such as <B>...</B> and <FONT>...</FONT> to present data. XML also uses labels. However the labels are user-defined (or "extensible") for it is the user who knows the data he/she has created. Also XML is more concerned with what the information means rather than knowing how to present it. This is an important step up from HTML in that it becomes possible for a computer (and people) to find the one piece of data in an XML document, handle it in a certain way, and use it for whatever purpose.

Understanding the limitations of HTML

Let us compare XML more closely to HTML. Suppose you created a table with data looking like so in HTML,

<TR>

<TD>Joanne Gayle Jones</TD>

<TD>08/28/1955</TD>

<TD>female</TD>

<TD>teacher</TD>

<TD>42,000</TD>

</TR>

<TR>

<TD>Ruth Ann Green</TD>

<TD>08/25/1978</TD>

<TD>female</TD>

<TD>teacher's assistant</TD>

<TD>21,500</TD>

</TR>

The first thing you will notice are the dates: how would a computer know for instance that 08/28/1955 is the date of birth of Joanna Gayle Jones or something else? This is the fundamental problem with HTML.

You could almost say you need another layer of elements on top of HTML specifically tailored to define the data stored in your table and not just a bunch of predefined elements for defining the way information should be presented.

To put it another way, suppose you needed to format just the date of birth and nothing else, leaving the rest of the text intact. In HTML, to format a piece of text would require you to manually go through the entire HTML document to put in your preferred HTML formatting code in the right location. Yuk! This is messy to say the least unless there is a way to define the data under a specific name you choose and use that name to apply the formatting, allowing every instance of the data defined by the name in an HTML document to instantly change or be updated.

XML is good at separating structure from the presentation. Because of this inherent advantage of XML to define and describe the information, it is fast becoming the standard for exchanging and displaying information on any kind of system.

Can we do anything we like with XML?

Not entirely. XML does have to be strict in the rules people must follow in order to keep things consistent and to minimise errors for all systems (e.g. people and computers). For example, you can't overlap tag boundaries like so as it would generate a parsing error,

<b><i>Unhappy XML</b></i>

However, the following is acceptable:

<b><i>Happy XML</i></b>

We call this a strict hierarchical nesting of elements. The hierarchy also helps to show the links between data. In other words, it gives you the opportunity to be a control-freak by getting your information thoroughly organised.

For example, suppose we had the following XML text called the "students" language:

<students>

  <student>

    <name>

      <first>John</first>

      <middle>Michael</middle>

      <last>Green</last>

    </name>

    <birth_date>

      <month>8</month>

      <day>28</day>

      <year>1992</year>

    </birth_date>

    <sex>male</sex>

    <grade>3</grade>

  </student>

</students>

The tags <first>, <middle> and <last> are nested inside <name>, which in turn is nested inside <student>. And <student> is nested inside <students>. This is what we call showing the hierarchy in the data.

Other rules of XML also include:

  1. Each piece of information you want to describe must start and end with a label (i.e. the XML element or tag).
  2. Each element starts with < and ends with >.
  3. The element that precedes a piece of information does not begin with a slash (/). But the element that follows a piece of information does have a slash.

Fortunately there aren't too many rules. And to make life easy there are software programs called XML Parser to tell you whether or not your XML is well formed. XMLwriter is one such tool.

Why do people use XML?

Because people haven't got better things to do with their lives other than sit in front of a computer. Seriously, we use XML to extract the key pieces of information or data (i.e. identify the important content) and give it a name. We call this the process of describing the information. In that way we can later use the name (in the form of an XML element or tag) to either display or print the data in a certain way or to exchange the data to help other systems use it to achieve something useful.

How did it come about?

In the early days of the printed text, people were more concerned with the content and rarely would they bother on good presentation. Whether it was a word etched on a stone rock or a message on a piece of skin from an animal, it was the intention of the author to get the content through to readers (apparently the author would like to let the part where we see people being chased by predators form the actual presentation whenever the content of the information is not read properly).

The 20th century saw an additional characteristic added to the content. It became quickly understood by publishers how important it is to focus on good presentation. For if publishers wanted to sell content to the mass market, readers demanded well presented content. If this wasn't true, then we would not have picture books or coffee-table editions containing high quality photographs and illustrations. And we wouldn't be bothered with choosing the right fonts and so on.

Then the internet came onto the scene in a big way during the 1990s as an alternative publishing medium. While it heralded a new era for the publishing movement, the presentation of information to the masses via the internet still played an important role thanks to the development of the HyperText Markup Language (HTML). It means people can use fixed HTML tags to present information in a rudimentary but often effective way on the electronic page of a browser, such as the HTML tag for changing the font typefaces via <FONT TYPE="Times, Arial, Helvetica">...</FONT>, and still get some enjoyment out of reading information online.

However, publishers (and their team of graphic artists) are rather fussy people. They want more control over the presentation of the content to the point where the quality of the output on a web page should be the same as the printed paper medium.

CSS (Cascading Style Sheet) improved the situation by helping publishers create user defined names for various styles of formatting or presentation schemes and use it within an HTML document to change the appearance of different information segments in a given content (e.g. titles, paragraphs, footnotes etc). These names allowed the power to change how the content of an entire web site looks through a careful selection of font types, font sizes, type style (bold, italic etc), line spacing, colour etc. Once defined, an HTML document with access to the CSS stylesheet can enact a defined presentation scheme within the stylesheet by using the <SPAN class=name>...</SPAN> HTML tag.

Thanks to CSS, publishers can now go a long way towards achieving the goals of producing a high-quality web page for everyone.

Then came a host of different presentation devices to keep our life entertaining such as digital TVs and WAP phones. It wasn't just a page in a web browser or the traditional paper medium which publishers had to handle. Now publishers had to create new HTML pages to make the content look good on all these different devices.

And the data (or content) portion of the HTML had to be duplicated for each page let alone for each presentation device. A rather messy and time-consuming idea.

Now publishers were looking for a way to separate the data from the presentation so that the presentation templates designed for different devices can point to the same data and present the data in the right way, saving them time and money.

Similarly, programmers with their software applications wanted to identify the data for sharing, sometimes in real-time, with other applications so as to avoid duplicating those functions through extra plug-ins or complex importing and exporting scripts.

This is where XML comes into the picture.

Another reason for XML — to avoid duplicating code

There is one other reason to use XML. It is to avoid duplicating slabs of HTML code for each and every page of a web site.

Take, for instance, the SUNRISE web page you are reading. While we have gone to some lengths to keep the HTML code relatively compact for easy downloading when generating this web page, it is still believed by others to be inefficient because we have to duplicate the HTML code for creating the bar effect on the left of this text, the buttons, logo and other paraphernalia for every web page. Somehow we need to separate this portion of the code from the rest of the page. And if we choose to change the design to suit different technologies such as WAP phones, then the actual data (i.e. what you are reading) would have to be separated to allow us to design different presentations before merging the data with our designs.

True, we could use frames to solve the duplicate HTML code. But not everyone likes frames. Not that it bothers us particularly very much. The browsers today are sufficiently capable of displaying frames. It is only those with small displays on WAP phones and PDAs who will hate us if we did apply frames throughout our web site.

Do I need to learn XML?

Excellent question!

To be truthful, you don't need XML for the moment. Any good flexible HTML design to handle different screen sizes and publishing mediums (i.e. look good when printed on paper and can be squashed down to fit on a PDA screen) together with the existing crop of web-enabled databases such as FileMaker Pro 4 to share data on the internet is enough to achieve 95 per cent of the aims of people going online.

But in the next 5 years you might.

Will XML replace HTML?

It will be several years before XML completely replaces HTML as the preferred format for creating Web pages. HTML is far too popular and useful. And more importantly, it is sufficiently easy to learn. But as the tools to make XML pages become more powerful and readily available and if they allow one to design a Web page (or a printed page) with ease without learning all the elements, HTML will eventually give way to XML. Just so long as the people who use it don't have to become real programmers.

But that will only come when the true power to present data well is finalised through another language called XSL.

Is XML really a language?

Strictly speaking, XML is not a language. It is a set of rules for explaining how to create a language to help us describe something. For example, you could use XML to describe the Spanish language like so:

<spanish>

  <hello>hola</hello>

  <bye>adios</bye>

  <book>libro</book>

</spanish>

But strictly speaking XML is not a language in itself. XML is a means of specifying how to create other languages.

Here is an analogy. Suppose you have two languages: English and Spanish. Each language has a set of common rules. Those rules include:

  1. A space separates each word.
  2. A group of words form a sentence.
  3. A sentence begins with a capitalised letter in the first word.
  4. A sentence ends with a period (.), question mark (?), or exclamation point (!).

Likewise, an XML document creates a language by describing a set of rules. For example:

  1. Each piece of information is contained within a label.
  2. Each label starts with < and ends with >.
  3. A slash (/) always begin in the label that follows directly after a piece of information.
  4. A piece of information is contained within a larger piece of information in a hierarchical manner.

But once I define the language through XML, then what?

Once you have the language defined through XML, it is possible to not only display pieces of information or content how you like it, but you can also let one application read the data in another application and vice versa so as to perform a variety of tasks such as displaying a calendar of information, translating a foreign sentence into English, or turning on a tap and perhaps even serve you beer.

Show me an example of a complete XML document

Here is an example of a complete XML document:

<?xml version='1.0' ?>

<students>

  <student>

    <name>

      <first>John</first>

      <middle>Michael</middle>

      <last>Green</last>

    </name>

    <birth_date>

      <month>8</month>

      <day>28</day>

      <year>1992</year>

    </birth_date>

    <sex>male</sex>

    <grade>3</grade>

  </student>

  <student>

    <name>

      <first>Jane</first>

      <middle>Marie</middle>

      <last>Boyd</last>

    </name>

    <birth_date>

      <month>2</month>

      <day>26</day>

      <year>1993</year>

    </birth_date>

    <sex>female</sex>

    <grade>2</grade>

  </student>

  <student>

    <name>

      <first>Nancy</first>

      <middle>Elizabeth</middle>

      <last>Butterworth</last>

    </name>

    <birth_date>

      <month>4</month>

      <day>05</day>

      <year>1993</year>

    </birth_date>

    <sex>female</sex>

    <grade>2</grade>

  </student>

</students>

Want more? Here is another example:

<?xml version='1.0' ?>

<?xml-stylesheet type="text/xsl" href="faculty_members_table.xsl"?>

<faculty_members>

  <faculty_member>

    <name>

      <first>Joanne</first>

      <middle>Gayle</middle>

      <last>Jones</last>

    </name>

    <date_of_birth>08/28/1955</date_of_birth>

    <gender>female</gender>

    <job_title>teacher</job_title>

    <salary>42,000</salary>

  </faculty_member>

  <faculty_member>

    <name>

      <first>Ruth</first>

      <middle>Ann</middle>

      <last>Green</last>

    </name>

    <date_of_birth>08/25/1978</date_of_birth>

    <gender>female</gender>

    <job_title>teacher's assistant</job_title>

    <salary>21,500</salary>

  </faculty_member>

  <faculty_member>

    <name>

      <first>Vincent</first>

      <middle>John</middle>

      <last>Pellio</last>

    </name>

    <date_of_birth>07/16/1959</date_of_birth>

    <gender>male</gender>

    <job_title>teacher</job_title>

    <salary>36,500</salary>

  </faculty_member>

</faculty_members>

This XML text may be stored in a text file called "faculty_members_table.xml". Notice the line <?xml-stylesheet type="text/xsl" href="faculty_members_table.xsl"?>. This line tells, say, an internet browser there is another text file called "faculty_members_table.xsl" which is needed to be processed if the user wants to present the data properly (probably for a given device such as an internet browser). We will discuss XSL in a little more detail later.

The above two examples are typical of the way we describe data in XML.

What is contained in this XML document?

We begin with <?xml version='1.0'>. This line tells any computer (including humans reading it) that what follows is an XML document and it conforms to version 1.0 of the XML specifications.

The tag <students>...</students> is the top of the hierarchy for this XML document. It tells a computer (or ourselves) that what follows will contain information to describe each student.

One student is described between <student>...</student>. It contains the first, middle and last name followed by the month, day and year of the student's birthdate, the gender (or sex) and grade. Between these labels contain the actual data.

As you can see, an XML document will always have a corresponding pair of labels such as <grade>...</grade>. A well-formed XML document must always have these pairs to open and close off sections of information.

This is just like HTML where you need a matching pair to close off a formatting option such as <B>...</B>.

Can I use any name for my labels in XML?

Yes you can! That's why it is called "extensible". However you might find it easier to check the thousands of standard XML definitions covering different subjects from astronomy, computers, commerce, genealogy and much more. It will make it just a little easier to exchange data if you do.

But how does a computer understand an XML document?

You have to create a text file called a DTD. DTD means Document Type Definition. You need this document to help a computer understand your XML document. You only need it if you want to get a computer to read and write data into the right locations in an XML document. Remember, you don't need it for merely displaying data (Phew! Less programming please!).

Here is an example of a DTD document:

<!ELEMENT faculty_members

  (faculty_member+)>

<!ELEMENT faculty_member

  (name, date_of_birth, gender, job_title, salary)>

<!ELEMENT name

  (first, middle, last)>

<!ELEMENT date_of_birth

  (#PCDATA)>

<!ELEMENT gender

  (#PCDATA)>

<!ELEMENT job_title

  (#PCDATA)>

<!ELEMENT salary

  (#PCDATA)>

<!ELEMENT first

  (#PCDATA)>

<!ELEMENT middle

  (#PCDATA)>

<!ELEMENT last

  (#PCDATA)>

How this works is that each time an ELEMENT (i.e. labels) tag is displayed, you define the XML data (ie, label) tags and any additional tags under the hierarchy. When you get to the lowest denominator for your data (i.e. the deepest level of your hierarchy), just say <! ELEMENT gender (#PCDATA)> for instance as this tells a computer data exists in the XML tag called <gender>...</gender> of the XML document.

If you see a "+" sign next to an XML tag name in a DTD document, it tells a computer to see the tag as something that must occur at least one or more times inside the "faculty-members" section.

If you see a "*" sign next to an XML tag name, it means it can occur zero or more times under this section.

The command #PCDATA means parsed character data. It tell the computer that the element contains data that can be parsed by a parser like HTML.

W3C has kindly formulated a new standard called XML Schemas. This will supercede DTD.

Still confused? Don't worry. It is highly recommended that you download tools to automate this process. One example is XMLSpy, especially for compiling XML Schemas with ease and in editing DTD documents.

***

What is XSL?

Ah! Now we come to another beast in this change to another language. XSL stands for Extensible Stylesheet Language. Again it is said to be the universal language for presenting data in an XML document on different publishing mediums or applications, including the internet browser. Presently XSL is most commonly used to create an HTML file with the help of an XSL processor such as the latest internet browser to help present data held in an XML document to users.

Why get XSL to create HTML?

Well, because XSL hasn't quite reached the level of power needed to present data in its own language on any device in the known universe. The people responsible for making XSL are still finalising a few things (God bless the guys for all their hard work). And then the internet browsers and other devices will have to be updated to handle the complete XSL when it eventually comes out.

Hence the reason why you shouldn't get too excited with XSL for the moment.

Where XSL fits into XML

Having XML and perhaps a DTD to communicate directly with computers and its applications is good and fine when it comes to identifying the data, describing it, showing how the data is related to each other through a hierarchical structure and communicating the data to machines. Unfortunately pure XML documents on their own aren't good at presenting the data in an attractive way for humans to understand.

On the other end of the scale, the problem with HTML/CSS is that it may be able to make data look very good if you wish, but it cannot perform calculations, personalize the data according to the user or session, or rearrange or sort the data in precisely how you or someone else likes it. Somehow there must be a way to combine CSS/HTML with XML to create the ultimate language for the web.

This is why XSL was invented.

Is XSL available now?

Yes, but in a limited form.

While XSL is designed to be used with XML, it is the aim of XSL to replace HTML and CSS.

When displaying data on the internet, XSL can be considered a kind of marriage between CSS/HTML (the presenting of data) and XML (the organising of data) where publishers can have their sophisticated formatting as well as the usual defining and organising of the data through XML. In that way, the correct data can be shared, manipulated and presented virtually in real time with any application.

In its current form (as of mid-2006), formatting data in XSL is a work in progress at W3C. But the first stage known as XML StyleSheet Transformation Language (or XSLT) is available now.

So what am I up against here?

When this whole situation with XML and XSL is completed and ready, you will face in the future the following files when setting up a web site:

datasource.xml + style.xslt=page.html

where "page.html" is generated by "style.xslt" after it was applied to "datasource.xml" to extract the data.

Look far enough into the future and the latest browsers should be able to work directly with XSL to create the web pages. No more page.html. All you need is the content stored in datasource.xml and a style.xslt to generate the web page for a web site.

How different is this from say, XML?

XSL is not unlike XML, except it has its own language for handling XML documents. Because XSL is not fully developed to handle all formatting to replace HTML and other languages, an XSL text document is also a kind of transitional stage document. It will contain, for instance, HTML or other foreign code to help perform the formatting on behalf of XSL.

Thus XSL will take any specific data you want (usually from an XML document) and place it inside, say, a piece of HTML code and an appropriate XSL processor such as a modern internet browser can processor the XSL code and create the HTML page from it with the data included so that it will appear as a well designed HTML/CSS web page.

But do remember one thing. For this technology to work on the internet, you must have the latest internet browser.

But how do I format an XML document to make it look good?

If you want to present the data described by XML on current internet browsers, you have to apply the XML-presentation technology known as Extensible Stylesheet Language (XSL) and make the transformation to HTML as required to present data on the web.

Further details on the language used is available below.

So what's good about XML/XSL if it is not for everyone?

Good question. The technology is mainly good for organisations such as a university or a corporation where everyone will have the latest internet browser and therefore can access the data efficiently on different technologies such as WAP phones and digital TV using different XSL documents. Also web sites are getting very big. The need to simplify the process is now greater than ever for large organisations.

For everyone else, the choice is either:

  1. Design a flexible HTML page using the TABLE tag to allow the presentation of data to look good on any display size using any technology; or
  2. Use a standalone XSL processor on the Web server to generate the HTML code and have it sent to the visitor's internet browser.

But soon, everyone will have the latest internet browser, XSL will be completed soon, and tools will be available to do the hard yakka in programming. Then the only question you will ever ask is what kind of design will work on all technologies to minimise the number of XSL documents to produce, and what you want to say to everyone using the technologies. And if you have to create different XSL documents, at least you will have the piece of mind of never having to duplicate your data. It will all be centralised and accessible from a single XML document.

If you are game enough to try your hand at XSL/XML and want to handle different browsers and technologies, consider the following XSL processors:

XMLWriter (Windows 95/98/NT/2000) $42

Apache Cocoon (Mac and Windows) Free

Tell me again, what is the difference?

Yeah, we know it is a bit of a bummer to understand XSL and XSLT stylesheets, but let's have a go anyway: An XSLT stylesheet is essentially an XML document containing labels starting with "xsl:" and works through a recursive matching and substitution in order to transform one XML document to another. For example,

  [Substitute with something else and continue]

The XSL labels may contain instructions for inserting data from an input document (e.g. XML) into something else or to calculate a value. And there is usually some foreign code to present the data until XSL is fully developed (e.g. HTML). But its main use is to access data from an XML document and present it in different ways to suit different applications or purposes.

For example, you can present the data in XML documents in Braille format. Or you could format XML information for presentation on Personal Digital Assistants (PDA) or WAP phones.

Example of an XSL document

To help you better understand the difference, here is an example of an XSL document:

<xsl:stylesheet xmlns:xsl="http://www.w3.org/TR/WD-xsl">

<xsl:template match="/">

  <xsl:apply-templates select="faculty_members"/>

</xsl:template>

<xsl:template match="faculty_members">

  <HTML>

    <HEAD>

      <TITLE>faculty_members data displayed in table format</TITLE>

      <BASEFONT FACE="Arial, Helvetica, sans-serif"/>

    </HEAD>

    <BODY>

      <TABLE>

        <TR>

          <TH ALIGN="LEFT">Name</TH>

          <TH ALIGN="LEFT">Date of Birth</TH>

          <TH ALIGN="LEFT">Gender</TH>

          <TH ALIGN="LEFT">Job Title</TH>

          <TH ALIGN="LEFT">Salary</TH>

        </TR>

        <xsl:for-each select="faculty_member">

          <TR>

            <TD>

              <xsl:value-of select="name/first"/>

              <xsl:value-of select="name/middle"/>

              <xsl:value-of select="name/last"/>

            </TD>

            <TD>

              <xsl:value-of select="date_of_birth"/>

            </TD>

            <TD>

              <xsl:value-of select="gender"/>

              </TD>

              <TD>

              <xsl:value-of select="job_title"/>

            </TD>

            <TD>

              <xsl:value-of select="salary"/>

            </TD>

          </TR>

        </xsl:for-each>

      </TABLE>

    </BODY>

  </HTML>

</xsl:template>

</xsl:stylesheet>

So when an XML parser loads up the XML document and an XSL processor runs this XSL document for presenting the data in the XML document (both the parser and processor are found in the latest internet browsers), a HTML web page is created as follows:

<HTML>

<HEAD>

<TITLE>faculty_members data displayed in table format</TITLE>

<BASEFONT FACE="Arial, Helvetica, sans-serif"/>

</HEAD>

<BODY>

<TABLE>

<TR>

<TH ALIGN="LEFT">Name</TH>

<TH ALIGN="LEFT">Date of Birth</TH>

<TH ALIGN="LEFT">Gender</TH>

<TH ALIGN="LEFT">Job Title</TH>

<TH ALIGN="LEFT">Salary</TH>

</TR>

<TR>

<TD>Joanne Gayle Jones</TD>

<TD>08/28/1955</TD>

<TD>female</TD>

<TD>teacher</TD>

<TD>42,000</TD>

</TR>

<TR>

<TD>Ruth Ann Green</TD>

<TD>08/25/1978</TD>

<TD>female</TD>

<TD>teacher's assistant</TD>

<TD>21,500</TD>

</TR>

<TR>

<TD>Vincent John Pellio</TD>

<TD>07/16/1959</TD>

<TD>male</TD>

<TD>teacher</TD>

<TD>36,500</TD>

</TR>

</TABLE>

</BODY>

</HTML>

Ostensibly what you are seeing here is the transformation of XML to HTML. In other words, you are looking at an XSLT stylesheet (until the time comes when XSL is fully developed with the labels to present data without the need for HTML).

Now suppose you wanted to present the data in a different way. Perhaps you want to say:

Joanne Gayle Jones is a female teacher who was born on 08/28/1955 and earns $42,000 per year.

Ruth Ann Green is a female teacher's assistant who was born on 08/25/1978 and earns $21,500 per year.

Vincent John Pellio is a male teacher who was born on 07/16/1959 and earns $36,500 per year.

What do you do?

Well, firstly you don't have to duplicate the XML document. The data you need is already there. So what you do is create another XSL document to present the data how you like it. Here is the new XSL document:

<xsl:stylesheet xmlns:xsl="http://www.w3.org/TR/WD-xsl">

<xsl:template match="/">

  <xsl:apply-templates select="faculty_members"/>

</xsl:template>

<xsl:template match="faculty_members">

  <HTML>

    <HEAD>

      <TITLE>faculty_members data displayed in sentence format</TITLE>

      <BASEFONT FACE="Arial, Helvetica, sans-serif"/>

    </HEAD>

    <BODY>

      <xsl:for-each select="faculty_member">

        <xsl:value-of select="name/first"/>

        <xsl:value-of select="name/middle"/>

        <xsl:value-of select="name/last"/>

        is a

        <xsl:value-of select="gender"/>

        <xsl:value-of select="job_title"/>

        who was born on

        <xsl:value-of select="date_of_birth"/>

        and earns

        $<xsl:value-of select="salary"/>

        per year.<BR/><BR/>

      </xsl:for-each>

    </BODY>

  </HTML>

</xsl:template>

</xsl:stylesheet>

This XSL document will generate the following HTML code for an internet browser:

<HTML>

<HEAD>

<TITLE>faculty_members data displayed in sentence format</TITLE>

<BASEFONT FACE="Arial, Helvetica, sans-serif" />

</HEAD>

<BODY>

Joanne Gayle Jones is a female teacher

who was born on

08/28/1955

and earns

$42,000

per year.<BR />

<BR />

Ruth Ann Green is a female teacher's

assistant who was born on

08/25/1978

and earns

$21,500

per year.<BR />

<BR />

Vincent John Pellio is a male teacher

who was born on

07/16/1959

and earns

$36,500

per year.<BR />

<BR />

</BODY>

</HTML>

You don't have to touch the XML document unless you wish to update the data by re-exporting the XML document. You only have to create the XSL document to suit a different presentation requirement.

What about sharing data with different applications?

Are there different XSLT stylesheets for different applications?

Yes, there are different XSLT style sheets handling different applications. You can download a stylesheet for Microsoft Excel, Rich Text Format (RTF), HTML, Scalable Vector Graphics (SVG), and many others. You can freely download a stylesheet online to suit your needs.

For those people developing FileMaker Pro databases, check the FileMaker XSLT Library for ready-made stylesheets.

What is contained in an XSLT stylesheet?

Because each application accepts data differently, an XSLT stylesheet will contain a set of template rules written in the form "if this condition is satisfied in the input, then generate the output".

Are all the template rules tested in an XSLT stylesheet?

Each template rule in the input XML document is not necessarily tested nor is it done sequentially. A template rule can decide which template rules should be scanned and checked before generating an output. Think of it as like a tree structure where each template rule is applied to a node on a tree, which in turn may decide which other template rules should be applied.

Who created XML and XSL?

Of course, you might be wondering who is responsible for this monstrosity? Well, XML and XSL was defined by the World Wide Web Consortium (W3C), the same guys who created HTML for the public. For example, Version 1.0 of the XSL language became a reality on 16 November 1999 as a recommendation. Now it is version 2.0 as more and more people are realising the usefulness of the language in achieving things.



***



WEB PUBLISHING WITH FILEMAKER SERVER ADVANCED (FMSA)

SHOWING THE POTENTIAL USEFULNESS OF XML/XSLT

Creating a simple XML query

Assuming your FileMaker Pro (.fp7) database file is properly set up for acepting XML queries (check the extended privilege set), you can type in a browser an XML query formulated into an HTTP request (i.e. a URL address) to help perform a certain task with the database.

For example, let's say your database is called "databasename.fp7" and you happen to know the IP address (or domain name assigned to the IP address) where FileMaker Server Advanced is hosting it. Suppose you want to insert a number like 500 into a field called unit in record 1. Then you like to go to layout 1. And while you are at it, why not execute a script called "doscript" all at the same time.

You can achieve all of this by typing in the browser:

http://<IP Address>/fmi/xml/FMPXMLRESULT.xml?-db=databasename&-lay=one&-script=doscript&-recid=1&-edit&unit=500

This is the fundamental command to manipulating a FileMaker Pro database in a web environment. Later we will provide the wrapping by way of an XSLT stylesheet to help display the results in a readable manner (in this case, HTML for displaying in an internet browser).

Understanding the structure of an HTTP request to FileMaker Server 8

To perform something (i.e. a query) on a FileMaker Pro 7/8 database, you type the following URL:

[protocol]//[server-ip][:port]/fmi/xml/[xml_grammar].xml?[query-string]

where [protocol] is either http or https depending on whether you have configured your Web Server for SSL. If so, use https, or else stick to the standard http.

[server-ip] is the IP address of the Web server. It doesn't matter if the Web Publishing Engine is on a separate computer. You must specify the Web server address.

[:port] is a port number you may wish to enter only if your Web server is using a port other than the defaults of 80 for http and 443 for https.

[xml_grammar] is one of four FileMaker XML grammar statements: FMPXMLRESULT, FMPXMLLAYOUT, FMPDSORESULT or fmresultset. Please note that these grammar names are case-sensitive.

And [query-string] is where you actually perform a task on your FIleMaker Pro database. It is here where you actually specify the command and one or more query parameters to help you perform a certain action. A query string must contain only one query command. Each query command can have several query parameters.

Here is an example:

http://www.webserver.com.au/fmi/xml/fmresultset.xml?-db=databasename.fp7&-lay=LayoutName&Field1Name=Michael&Field2Name=Smith&IDField=123&-new

This URL tells the FileMaker Server (through the Web server and Web Publishing Engine) to create a new record and place the data "Michael" in the field Field1Name and "Smith" in the field Field2Name and "123" in the field IDField which exists in the layout called LayoutName of the database called databasename.fp7.

Here is another example. Suppose you typed:

http://www.webserver.com.au/fmi/xml/fmresultset.xml?-db=products&-lay=sales&-findall

This URL tells the FileMaker Server to find all records in the products.fp7 database using the fields in the layout called "sales". The result is displayed in the browser as an XML document looking something like:

<fmresultset version="1.0">

<error code/>0

<product build="[date]" name="FileMaker Web Publishing Engine" VERSION="8.0.4.128"/>

<datasource database="products" date-format="MM/dd/yyyy" layout="sales" table="[table name for storing sales records in the database]" time-format="HH:mm:ss" timestamp-format="MM/dd/yyyy HH:mm:ss" total-count="310"/>

<metadata>

<field definition auto-enter="yes" global="no" max-repeat="1" name="[field name 1]" not-empty="no" result="date" type="normal"/>

[Repeat for other field names]

</metadata>

<resultset count="[Number of records found]" fetch-size="[How many to display in one go]">

[Shown here would be a series of elements followed by the field data shown between ]

</resultset> </fmresultset>

and consists primarily of the elements , and .

The element generatred by the Web Publishing Engine provides the essential information about the name of the database, layout, table containing the data, total count, and format for date and time as well as other database attributes.

The element defines the fields and their attributes.

The element returns the result after applying the query to the database via the above URL.

If you look at the source code for this page you will see additional information at the start. They are:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>

<!DOCTYPE fmresultset PUBLIC "-//FMI/DTD fmresult//EN" "/fmi/xml/fmresultset.dtd">

<fmresultset xmlns="http://www.filemaker.com/xml/fmresultset" "version="1.0">

Alternatively you could use FMPXMLRESULT in the grammar. It is functionally equivalent to fmresultset. For example:

http://www.webserver.com.au/fmi/xml/FMPXMLRESULT.xml?-db=products&-lay=sales&-findall

will generate an XML document in the browser looking like:

<FMPXMLRESULT>

<ERRORCODE>0</ERRORCODE>

<PRODUCT BUILD="[date]" NAME="FileMaker Web Publishing Engine" VERSION="8.0.4.128"/>

<DATABASE DATE FORMAT="MM/dd/yyyy" LAYOUT="sales" NAME="products" RECORDS="[total number of records]" TIMEFORMAT="HH:mm:ss"/>

<METADATA>

<FIELD EMPTYOK="[YES or NO]" MAXREPEAT="1" NAME="[FIeld Name 1]" TYPE="[Field type eg. DATE, TEXT, TIME etc]"/>

[Repeat for other field names]

</METADATA>

<RESULTSET FOUND="[Number of records found]">

[Shown here would be a series of <ROW> and <COL> elements followed by the field data shown between <DATA></DATA>]

</RESULTSET> </FMPXMLRESULT>

Note how the XML document results page shows the errorcode. If there are no errors during the running of the query, the result will be 0. If it isn't, the number generated will be the same as those generated by a standard FileMaker Pro database in the scripting environment. For example, error code 802 means "Unable to open file" and 202 means "Field access is denied" (check your FileMaker Pro help section).

So what does FMPXMLLAYOUT grammar do? You use this to see the value lists and any information about the fields used in a given layout. Choose -view query command in the URL to request the XML document to be generated from the Web Publishing Engine. For example,

http://www.webserver.com.au/fmi/xml/FMPXMLLAYOUT.xml?-db=products&-lay=sales&-view

An XSLT Alternative

Think this is too taxing on the fingers to type? Try XSLT. Assuming you have created an XSLT stylesheet text file containing the following code:

<?xml version="1.0"?>

<?xslt-cwp-query params="-grammar=FMPXMLRESULT&-db=databasename &-lay=nylayout&-script=doscript&-recid=1&-edit"?>

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:fxr="http://www.filemaker.com/FMPXMLRESULT">

<xsl:output method="xml" />

<xsl:template match="@*|node()">

<xsl:copy>

<xsl:apply-templates select="@*|node()"/>

</xsl:copy>

</xsl:template>

</xsl:stylesheet>

saved it as, say, "query.xsl" and store the file in the folder \FileMaker Server 8\Web Publishing\xslt-template-files\ on the server hosting the database, then it is possible to type this URL:

http://<IP Address>/fmi/xsl/query.xsl?unit=500

The URL shown here will perform the exact same function as the XML query we typed earlier. But remember to set up the FileMaker Databases for XSL queries (check the extended privilege set, or see below).

How to set up FileMaker Databases for XML queries

A. FileMaker Server Advanced (FMSA)

  1. Under the Console Root folder in the left pane, open up Windows Administrative Tools.
  2. Click on "Services (Local)".
  3. The services you are looking for are "FileMaker Publishing Engine (CWPE & CONFIG)" and "FileMaker Publishing Engine (WPC)". If these services are available, the Status column should indicate the services have been started. If not, right click on each service and choose "Start" from the pop-up menu.



B. FileMaker Pro 7/8

  1. Open your FileMaker Pro database.
  2. Choose File>Define>Accounts & Privileges
  3. If you haven't done so already, create the account that will allow users access to the database via XML. If you have accounts already established, choose the one you will use for XML access and click the Edit button.

  4. In the Edit Account window, click Edit button next to "Privilege Set".
  5. In the Edit Privilege Set window, you will see "Extended Privileges". Put a tick in the check box that says "Access via XML Web Publishing - FMSA only (fmxml)".

  6. Click OK all the way through to exit Accounts & Privileges. You will be asked to type the username and password of the administrator of the database for the above changes to be accepted.
  7. Close the database.



C. Publishing the database on FMSA.

  1. Move the XML-enabled database to the location where FileMaker Server Advanced can host it. This would be \FileMaker\FileMaker Server\Data\Databases\.

  2. In FileMaker Server Advanced, choose Console Root\FileMaker Server\Databases\Folders\.
  3. Right click any where in the right pane and choose Refresh.
  4. Right click on your database and choose Open. The status should indicate the database is opened (i.e. Normal).

  5. A special note for MacOSX users: You should manually set the UNIX permissions for your database to:

    Owner: fmweb

    Access: Read & Write

    Group: fmsadmin

    Access: Read & Write

    Others: Read only

Once your FileMaker Pro database has been set up correctly, you can happily send XML queries to your heart's content.

How to set up FileMaker Pro databases for XSL queries

Repeat the above steps for setting up a Filemaker Pro database for running XML queries except choose XSL instead. For example, in the Extended Privileges section, put a tick in the check box that says "Access via XSLT Web Publishing - FMSA only (fmxslt)".

NOTE: It is customary to enable fmxslt and fmxml extended privileges at the same time for a single FileMaker Pro account.

Where does XSLT stylesheets go?

If they are for FileMaker, drop the stylesheets into:

\FileMaker Server 8\Web Publishing\xslt-template-files\

on the server hosting the database

Give me a summary!

Here are the essential steps:

  1. In the Web Publishing Engine Administration Console of FileMaker Server, turn on XSLT publishing.
  2. In the Accounts and Privileges section of your FileMaker Pro databases, turn on fmxslt and fmxml extended privileges for an account you will use to web publish your databases.
  3. Type the text file known as an XSLT stylesheet. It contains the FileMaker-specific XSLT extension functions, query commands and query parameters you'll need to manipulate your databases and display the results received in XML into a readable format (e.g. HTML on a browser). Need help to create an XSLT stylesheet? Try your hand with the FileMaker Site Assistant tool for creating basic XSLT stylesheets as a starting point. Do you have older web pages containing CDML commands for manipulating FileMaker Pro 4-6 databases? Convert them to XSLT using the CDML Converter tool. Once you have a basic XSLT stylesheet developed, you can try your hand at modifying XSLT stylesheets using a dedicated XSLT authoring tool or use your own text editing tools (e.g. TextEdit or BBEdit).
  4. Move the XSLT stylesheets into the folder \FileMaker Server 8\Web Publishing\xslt-template-files\. This is important if FileMaker Server is to know where and make secure the relevant XSLT stylesheets. NOTE: XSLT stylesheets can be placed inside folders within the xslt-template-files folder.
  5. Place static files on the web server. These are usually web pages providing an introduction before users begin querying the databases using XSLT stylesheets. The static page can either provide a link to an XSLT stylesheet for users to click on. Or you could make the static page autoload into an XSLT stylesheet without intervention from the users.
  6. Check your security mechanisms for your site or program.
  7. Finally, make the site or program available to your users (i.e. the production phase).

NOTE: You should do all this on a development server before going into production. It is a safety measure in case the XSLT stylesheets are not doing the right thing or you need to adjust the security mechanisms for your site or program.

If all goes well, your stylesheets should display the results correctly and be able to dynamically obtain data from a FileMaker Pro database whenever a user sends the HTTP request and the appropriate URL to a specific stylesheet for transforming and formatting the XML data results to a readable form (usually as an HTML page).

What's the difference between Instant Web Publishing (IWP) and XSLT/XML web publishing?

Good question. IWP is good at displaying with reasonable accuracy the layout design and fields of a FileMaker Pro database on a web page. Use it to reproduce your layout designs for web publishing and perform basic tasks in a non-dynamic way (e.g. displaying or collecting static information such as an online form). But if you want to do more, for example, make your data more dynamic (e.g. displaying movies and sounds), and custom design your web pages showing only the data you want, then use XSLT/XML web publishing.

A common problem with Instant Web Publishing — missing portal data

There are some tips to remember when publishing data in IWP. The most useful tip is perhaps making sure you have set privileges to allow data in a field to be displayed in a web browser.

Here is a common request by one user trying to figure out why portal data does not display in Instant Web Publishing:

"...I can't get portal data to show up when the database is shared using Instant Web Publishing. Even though there are several rows of data in a record, the portal just looks blank when viewed via a browser (and it doesn't matter which browser we use). I'd be very grateful for any help." (http://filemakertoday.com/com/showthread.php?t=11172)

The solution is to go into Accounts and Privileges under the File menu of FileMaker Pro for your database. Edit the Instant Web Publishing user account (i.e. your online users). In Record privileges, make sure the fields of your portal data are set to "View".

That's it!

How useful is the FileMaker Site Assistant tool?

The FileMaker Site Assistant tool for generating FIleMaker XSLT stylesheets is useful as a starting point to building XSLT stylesheets. The Site Assistant tool can create a site to allow users to:

  • browse a single record at a time
  • view a list of all the records in the database
  • search the database and view the results in a list
  • sort records
  • add records
  • edit and duplicate records
  • delete records
  • view a summary report

and generate a home page to link to the other XSLT stylesheets generated by the tool.

NOTE: The Site Assistant builds stylesheets to transform FileMaker XML data into HTML pages based on the fmresultset XML grammar. Also remember that the SIte Assistant prefers to work with data from the same table in your FIleMaker Pro database. For example, if you have multiple tables and you select a layout to conduct, say, a search page, an edit records page, and/or an add records page which accesses a different table, unexpected results can occur. Make sure the layouts are from the same table. And check the utilities.xsl stylesheet for defining errors and common XSLT templates that are called by your generated stylesheets.

How do I use the CDML Converter tool?

The first thing to keep in mind is how this tool is not the perfect conversion tool. It is a useful tool only in the sense that it begins the process of converting CDML format files to XSLT stylesheets. However, under no circumstances should you see the tool of being able to perfectly convert CDML format files.

You need to be an experienced CDML developer (the old web publishing FileMaker commands) and have a good grip with Custom Web Publishing with XSLT (the new web publishing FileMaker commands).

Once you come to grips with CDML and XSLT, these are the essential steps to converting files with CDML Converter:

  1. DO NOT WORK ON THE ORIGINALS. Always make a copy of the original "production environment" (i.e. live) CDML format files to another location for you to work from.
  2. The CDML format files should be stored inside a source folder. Next, create a destination folder to store the XSLT stylesheets to be generated by the CDML Converter tool.
  3. Bring out your CDML Converter tool and begin converting the copy version of your CDML format files. The converted files will have the same filenames as the CDML format files except the file extension will be changed to ".xsl". One word of warning: if the source folder contains two files having the same filenames such as search.htm and search.cdml, both will be converted but it is the second file that is likely to overwrite the first converted file. To avoid this situation, try renaming these files to have a unique filename. And check you have not lost any references to those newly-named files in the CDML format files before conversion. Also choose the same text encoding as the source format files.
  4. Examine the stylesheets generated and conversion log (known as cdml2xsl_[datetime].log. With a bit of luck, the converted XSLT stylesheets may work on the latest FileMaker Server without any further modification. But sometimes you may get an error. Usually it is because a CDML tag couldn't be converted to an appropriate statement within an XSLT stylesheet. If this happens, it is time to do some manual editing. But even if you don't have errors, you may still need to do some editing. For example, a FileMaker date or time extension function inside an XSLT stylesheet may be formatted as MM/dd/yyyy for the American environment. Change it to say dd/MM/yyyy to suit your area. Likewise, where an XSLT stylesheet references an image file called, say, logo.jpg by using the HTML tag , the location will now be /fmi/xsl/logo.jpg on the web server. And does your FileMaker Pro database reference a CDML format file? If so, manually edit these references in the database to point to the correct converted XSLT stylesheet. In most cases, it may be a simple matter of changing the file extension to ".xsl" for XSLT stylesheet files and nothing else.

If everything is okay and there are no errors, move the converted XSLT stylesheets to the folder \FileMaker Server 8\Web Publishing\xslt-template-files\. You can also create a [folder] in here to store the XSLT stylesheets in case you have other databases needing their own XSLT stylesheets.

To request and process an XSLT stylesheet, use within your web pages (or type in the browser address field) the URL with syntax:

[scheme]://[filemakerserverhost][:port]/fmi/xsl/[folder]/[stylesheetname].xsl[?query string]

NOTE: Just like the FileMaker Site Assistant tool, the CDML Converter tool builds stylesheets to transform FileMaker XML data into HTML pages based on the fmresultset XML grammar.

The sort of problems you may encounter after conversion

Hopefully there won't be too many errors. But if you do encounter them, the following may help to bring your stylesheets to a healthy state:

  1. When I open an XSL file on a browser, I get the message: "FileMaker Custom Web Publishing with XSLT: Open quote is expected for attribute 'class' associated with an element type 'a'."

    Solution: Use BBEdit or some other text editor to show line numbers in the offending XSL file. This is useful as the error meesage will tell you the row and column number where the error was found. When you find the location, you might see the following:

    <a href="http://www.widgets.com/search" class=="class" whitehoverunderline="whiteHoverUnderline" "="":" accesskey="5">Search Page</a>

    However if you look elsewhere for similar coding, you might find:

    <a href="http://www.widgets.com/home" class="whiteHoverUnderline" accesskey="5">Home Page</a>

    This will give you a clue. Apparently a character or something had confused the CDML converter, so you must change the code to:

    <a href="http://www.widgets.com/search" class="whiteHoverUnderline" accesskey="5">Search Page</a>

    If you are not sure, look at the original CDML format files. Sometimes the original non-CDML related coding is okay to use in the XSL file (remembering of course that the code must satisfy the XML standards such as closing off a label or tag etc).

  2. When I open an XSL file in a browser, I get the message: "FileMaker Custom Web Publishing with XSLT: Element type 'option' must be followed by either specifications, '>' or '/>'. (XML)"

    Solution: Go to the line number for the error in the XSL file. You may notice something like this:

    <select name="Organisation Type" size="1">

    <option value=""> - No Selection -

    </option><option other="Other">Other

    </option></select>

    Again the converter was confused by the fact that there was no closing </option> tag in the original CDML format files. Also the value inside the option tag for Other looks kind of peculiar.

    The clue to solving this error is to look at the original CDML format files. Firstly you will notice other= should, in fact, be value=.

    Now you can fix this error by simply adding the closing option tag in the correct position and fix up the value like so:

    <select name="Organisation Type" size="1">

    <option value=""> - No Selection - </option>

    <option value="Other">Other</option>

    </select>

    Should you have to fix up lots of option coding because the pop-up list for the selected field name is long, you might be better off going to the original code in the CDML format files and using BBEdit or Microsoft Word to search and replace key repeating characters with the correct pieces of code to help speed up the process.

  3. When I open an XSL file in a browser, I notice a textarea box for a field filled with what appears to be XSL code when really it should be blank. Why?

    Solution: Here's another beauty you would be scratching your head for a long time unless you know what you are doing. The problem concerns your browser and how its built-in XSL processor gets confused when looking for a character to properly complete and render the <textarea>...</textarea> correctly on the page.

    For example, you might have in your code:

    <textarea name="Organisation Summary" cols="50" rows="5">

    But the browser was expecting a closing </textarea> or something like:

    <textarea name="Organisation Summary" cols="50" rows="5" />

    To solve this problem, consider typing the following:

    <textarea name="Organisation Summary" cols="50" rows="5">&#160;</textarea>

  4. When I open an XSL file in a browser, I notice the following error: "No XML grammar was specified in the "-grammar" query parameter (QUERY-ER0001)". Why?

    Solution: Your XSL file is missing a line near the beginning. The line is, "<?xslt-cwp-query param="-grammar=fmresultset"?>". Note that the grammar fmresultset could be different — check your other XSL files for a clue. An XSL file usually starts off like this:

    <?xml version="1.0" encoding="ISO-8859-1"?>

    <?xslt-cwp-query param="-grammar=fmresultset"?>

    Just put in the missing line and miraculous the error message will disappear.

  5. When I open an XSL file in a browser, I get the message "Prefix must resolve to a namespace: fmq (XSLT)". Why?

    Solution: We recommend that you type near the beginning of the XSL file the following namespace declaration statement:

    xmlns:fmq="http://www.filemaker.com/xml/query" within the <xsl:stylesheet version="1.0"...> label, and add directly after this label:

    <xsl:param name=request-query"/>.

    It is quite likely you are using $request-query in the body of your XSL file which hasn't been properly declared at the beginning.

  6. Other errors you might get include adding a space character between the end of a text quote character and the next character. For example, you could get:

    <select name="Organisation Type"size="1">

    If you are not proficient in XSL, you could be scratching your head for hours (which might explain why so many programmers go bald early in life) until you realise you only need to add the space character like so:

    <select name="Organisation Type" size="1">

    It is silly things like this that you have to be prepared for!

  7. Question: When I open an XSL file in a browser, I notice the following error: "xsl:comment is not allowed in this position in the stylesheet! (XSLT)". Why?

    Solution: It is probably because the <xsl:comment> is hanging on its own in a separate line and the XSL processor inside your browser or wherever doesn't know what this tag is doing. You might be better off removing all comment tags.

  8. Even after doing all of this and the XSL files appear to be generating HTML for a browser (i.e. you can see something reminding you of the original web pages), be aware that sometimes they won't appear exactly as in the original HTML files. For example, an original HTML file may required the DOCTYPE at the beginning to help render a page correctly in a browser. As the XSL file may show at the beginning:

    <! CDML CONVERTER WARNING: The tag "DOCTYPE" has been removed. >

    adding DOCTYPE will not work as XSLT spits a dummy. You may have to redesign the web page in HTML without the need for a DOCTYPE.

    Also, you may see something like this:

    <! CDML CONVERTER WARNING: A CGI query on this page was missing the -lay parameter. A -lay parameter has been added with the value 'AllFieldsLayout'.>

    This means the original coding was not explicit enough. Even though it may have worked before because you probably had just one layout in your FileMaker Pro database, you have to be more specific in XSLT. For example, the original code might look like:

    <a href="http://myserver.com.au/FMPro?-db=tsu_news.fp5&-Format=search_results.htm&-SortField=_date_created&-SortOrder=descend&-Max=5&-FindAll">Find all</a>

    Notice the missing -lay in this CGI code? What XSLT really wants to see is something like this:

    <a href="http://myserver.com.au/FMPro?-db=tsu_news.fp5&-lay=News&-Format=search_results.htm&-SortField=_date_created&-SortOrder=descend&-Max=5&-FindAll">Find all</a>

    where News is the name of the layout in your database called tsu.news.fp5.

    When converted to XSLT, it will probably look something like this:

    <a><xsl:attribute name="href">http://myserver.com.au/fmi/xsl/search_results.xsl?-db=tsu_news&;-lay=News&;-max=5&;-sortfield.1=_date_created&;-sortorder.1=descend&;-findall</xsl:attribute>Find all</a>

    Fun, isn't it?

Should I create an HTML home page to link to an XSLT stylesheet or create an XSLT home page?

It is considered good practice to include with your XSLT stylesheets a home page created in XSLT. The XSLT home page is one that doesn't require the user to type a query string (and thus will probably not prompt the user to enter a username and password to access data in a database).

Use the FileMaker Site Assistant tool to create an XSLT home page. You will be able to see how it works.

If you do create the XSLT home page and put it with the rest of the stylesheets in the same folder location, you can open the home page by typing:

http://[FileMakerServerIPAddress]/fmi/xsl/my_templates/home.xsl

How do I customise the iwp_auth.html and iwp_home.html web pages generated by FileMaker Server?

Don't like the web page design of iwp_auth.html and iwp_home.html? Well, you're not alone! And more importantly, you can do something about it. We have extracted the essential javascript code in the header sections of each file together with the relevant body section code for you to drop into your own customised web pages. When you are ready, drop your files into the appropriate IWP folder location. For details, click here.

How do I get users to automatically go straight into my database solution in IWP?

A fairly common request solved in the following way:

  1. Make sure you have a [Guest] account in your database solution. If need be, create it and define the privileges you will permit guest users to do in your database.
  2. In the Extended Privileges, put a tick in the boxes that say "Access via FileMaker Network (fmapp)" and "Access via Instant Web Publishing (fmiwp)".
  3. In all other accounts, turn off IWP in the extended privileges section. You only need one in the [Guest] account.
  4. Save the account details and close the database.
  5. Upload your database to the FileMaker Server folder for storing all web-enabled databases.
  6. Open the database in FileMaker Server to serve it on the network
  7. Open a web browser and type http://[yourServerAddress]/fmi/iwp/ and click the name of the database. You will go straight into the database without authentication.

If you don't want to see FileMaker's own iwp_home.html for listing your web-enabled databases, type a link within your own web pages to bypass it altogether, http://[yourServerAddress]/fmi/iwp/cgi?-db=[nameofdatabase]&-startsession.

That's all there is to it!

The problem with XML and XSL queries - poor security

You will notice that every time you type an XML and/or XSL query in the browser address field, you will be prompted to enter your username and password to access the appropriate database account before the query can be performed. Your username and password are not encrypted or hidden from others when the information is transmitted.

One solution is to set up XML and XSL extended privileges to the Guest account. Why? Because the Guest account will not prompt the user for authentication information. But this lays open your database to misuse from unwanted users.

Another solution involves appending the authentication information consisting of <Username> and <Password> to the IP address in the URL. For example, in XML, you could type:

http://<Username>:<Password>@<IP Address>/fmi/xml/FMPXMLRESULT.xml?-db=databasename&-lay=one&-script=doscript&-recid=1&-edit&unit=500

But again the risk here is that the transmission of the URL over the internet could be intercepted by a third-party. Unless the URL starts with https:// you would be risking having your database accessed by unwanted users over the internet in next to no time.

The security problem can be solved through one simple step. Try embedding the username and password in the XSL file and putting the file in the designated FileMaker Server folder for all XSLT stylesheets (i.e. the xslt-template-files folder). Because the XSL file is closely guarded by FileMaker Server within this folder and users can never see the source code of any XSLT stylesheet as a result, it is okay to store the username and password information inside the stylesheet.

Thus in the above XSL code, we can do the following:

<?xml version="1.0" ?>

<?xslt-cwp-query params="-grammar=FMPXMLRESULT&-process"?>

<xsl:stylesheet version="1.0"

xmlns:xsl="http://www.w3.org/1999/XSL/Transform"

xmlns:fxr="http://www.filemaker.com/FMPXMLRESULT"

xmlns:fmq="http://www.filemaker.com/xml/query">

<xsl:output method="xml" />

<xsl:param name="request-query" />

<xsl:variable name="myunit" select="$request-query/fmq:query/fmq:parameter[@name='unit']"/>

<xsl:variable name="xmltree" select="document( concat( http://accountname:password@localhost/fmi/xml/FMPXMLRESULT.xml?-db=databasename&-lay=one&-script=doscript&-recid=1&-edit&unit=', $myunit) )"/>

<xsl:template match="/">

<xsl:copy-of select="$xmltree"/>

</xsl:template>

</xsl:stylesheet>

So now, everytime you type:

http://<IP Address>//fmi/xsl/query.xsl?unit=500

the XSL query is run via the stylesheet called query.xsl using the authentication information supplied in the XSL file.

A more secure method

A more effective means of securing your database is to employ a second FileMaker Pro database, save it as a FileMaker runtime application using FileMaker Pro Advanced, host it on FMSA, and use within this second database an XML data import script to load FileMaker data from the first database. Because the second database is a runtime solution, administrative access has been removed. So there is virtually no way a user can ever access the second database as an Administrator. But all queries can be performed on this loaded data through the second database before it is returned to the first database in a secure manner.

Actually this second database can be used in a similar way to load and perform tasks on data in any scriptable XML application such as Microsoft Excel 2004.

One restriction with XSL/XML in FileMaker

You should be aware of one important limitation of FileMaker when performing any kind of web publishing. The -script command used in the above XML/XSL code may allow you to execute certain scripts developed within your FileMaker Pro databases. But not all FileMaker script steps are executed in the web publishing environment.

Some script steps may be described as "web-compatible" and therefore will proceed well in XML/XSL when run. But other scripts will not be compatible. This could pose a problem for the security of your database and create other issues such as the way the script manipulates and changes the data in the records. To check the scripts work properly, enable "Show web compatibility" during the script designing phase. Scripts that are incompatible will be greyed out. To see the exact behaviour of the script, make a copy of the database and the script itself, delete the web-incompatible script steps and run the entire script as is. What happens?

If the results are exactly what you want, then no problems at all. But if you find the script is not performing correctly, this is the time to improve the script for greater web compatibility (or wait until FileMaker Inc finds a way to increase the compatibility in the next version of FileMaker Server which could take years).

External server authentication

An important part of any FileMaker Pro databases is the ability to restrict user access through authentication (i.e. a user must type a username and passwor). If you choose to restrict your database, you will need to set up what is known as an Account. An account is just a way to insert a switch in the database to force users to enter a username and password. And if the information matches correctly, the user has been authenticated allowing them access to the records in your database.

There is also an extra level of security called a Privilege Set. This determines the level of authorisation you want to give to users when accessing information in your database. In other words, do you want users to be able to export data, delete records, edit records and so on. Or do you want to restrict them in what they can see and do? Either way, you can set the privileges users can have within an account by clicking the Edit button under Privilege Set in the Edit Accounts window.

Part of this Privilege Set also includes the Extended Privileges. From here you can turn on additional access points to the database via a network such as web publishing your database for users to access using an internet browser (for example, you can have Instant Web Publishing or IWP and XSLT/XML which negates the need for users to have a copy of FileMaker Pro to view your database) or through a simple FileMaker network or ODBC (which requires users to have a licensed copy of FileMaker Pro or a third-party application that can read ODBC data).

The standard "FileMaker Account" consists of the Account Name and Password. You specify account name and password through File>Define>Accounts & Privileges and specifying "FileMaker" under "Account is authenticated via:" as the method of authentication.

However, in organisations where people may already have authentication through an external server to acces their own PC or Mac, remembering another account name and password may be inconvenient. And anyway, a person who has already authenticated on a PC or Mac shouldn't have to remember another password. Likewise a developer of a FileMaker database shouldn't have to recreate every single account for each person if it has already been done by another administrator. This is where External Server authentication comes into the picture.

By selecting "External Server" as the method for authentication, you will let another application called FileMaker Server handle the authentication for you, which in turn will check the authentication on another more powerful account available through Windows OS and Mac OS X. These are called Active Directory Accounts for Windows and Open Directory Accounts for Macintosh. Because these OS level accounts are managed by another administrator, you don't have to set up new accounts in FileMaker for each user wanting to access the database. Everything is handled outside. All you have to do in your FileMaker Pro database solutions is check the Account Name that comes through from FileMaker Server via Get(AccountName) to determine if you want to give the user access to the database.

When selecting "External Server" as your preferred authentication, there is a crucial setting you must do in the Privilege Set. Make sure a tick appears in the Extended Privileges under "Access via FileMaker Network (fmapp)". Otherwise your FileMaker database will not appear in the list of hosted files and therefore will not be accessible to anyone.

Another crucial role in external authentication is how you order the list of Accounts in your FileMaker Pro database. There are consequences in moving Accounts around. For now let us just say that accounts to be authenticated by an external server should normally appear at the top of the list before the list of FileMaker Accounts with the exception of the one FileMaker Account that gives you full access to layouts, scripts etc. The [Guest] account usually appears right at the bottom and is usually disabled when external server accounts are established and active.

NOTE: Later you will realise that users can be part of many different groups (i.e. domains) on a network. For example, if you have sufficient administrator rights on your PC, you should be able to go: Start>Control Panel>Administrative Tools>Active Directory Users and Computers. A window will pop up showing the Active Directory users and computers in the left pane. Opening up these group folders until you reach the list of Users will tell you which user is a member of which group. Other groups a user might be in can be determined by double-clicking on the name of a user and choosing the Member Of tab. Hence these groups serve another function within FileMaker. The username you specify for External Server authentication in FileMaker can include the domain group so you can restrict access to a FileMaker database to only those users of the group. It means any user not part of the group cannot enter their active directory ID and password and expect to get in unless they are part of the group. To specify the group in FileMaker, type the group name as you see it in "Active Directory Users and Computers" in the Username. If the group called Group1 is embedded in another group called Group2, you would type "Group2\Group1" (without quotation marks). Then the user has to type their username ID and password and authenticated externally to see if the user is part of the group and has provided the correct username and password. If so, FileMaker Server gives the username of the user to the FileMaker Pro database (accessible via the function Get(AccountName)). From there you can decide how much further control you want to give to the user once inside your database through: (i) the privileges you have specified for the user in Accounts and Privileges; and (ii) the scripts you develop to make use of Get(AccountName) for that extra level of refinement.

You should also set up an "fmsadmin" account having full access to the privilege set and is authenticated by an external server.

Do you already have FileMaker Accounts present in the database? Make them inactive. It is likely the same users will already gain access to the database through the external server authentication system. Also leave active the administrator/developer "full access" FileMaker Account in case you need to go back into the database to make further changes to scripts, layouts and so on in the future.

NOTE: You can also use External Server authentication to give one or more accounts "full access" to a FileMaker Pro database.

After setting up your FileMaker Pro database to the way you want it, now we must go into FileMaker Server to tell it to accept External Server authentication. In Console Root in the left pane, the name of the FileMaker Server you are running appears below "FileMaker Server". Right click on it and select Properties. Click the Administration tab. In Administrator Authentication, click the radio button that says "Require membership in "fmsadmin" group. And also put a tick in the check box that says "Allow remote users to administer FileMaker Server". Click Apply button. Click OK. This will permit users of the fmsadmin group to perform administration tasks in FileMaker Server.

Next comes the client authentication side. Rick-click again to get back to the Properties window. Click the Security tab. Under Client Authentication, click the radio button that says "FileMaker and External Server accounts". Click the Apply button and click OK.

Next, you will have to know whether your PC is part of a domain or group. On a PC, Start>Control Panel>System. Click the Computer Name tab. You should see below the Full computer name whether or not you are part of a workgroup or a domain.

Workgroup PC

Is your PC on a workgroup? You have the option to use your PC running FileMaker Server to take charge of authenticating users in which case you need to have Administrator privileges to your PC where you can set up user Accounts and Groups on that computer forming your Active Directory. Assuming you have administrator access, right-click on My Computer and select Manage. From there you can set up users and a group name.

In Mac OSX, you achieve the same thing by using the Accounts preference pane for setting up user accounts of the people who will have access to your database. Use the OSX utility NetInfo Manager to create a Group name for the user accounts. Finally the OSX utility Directory Access (via the Authentication tab) will help you to get your FileMaker Server computer to use only the local Accounts and Groups. Set to "Local Directory" in the "Search" value list and click Apply for this to work.

Domain controller

However, in most cases, the authentication is usually handled outside the PC by what we call the Domain Controller. In this scenario, you only have to know the name of the domain. NOTE: Don't run FileMaker Server on the domain controller computer. It will dramatically reduce FileMaker Server performance when the domain controller is busy. Keep FileMaker Server on a separate computer.

Should you find your PC is not on the right domain, click the Change button in the System control panel under Computer Name tab.

We are almost finished!

If everything has been set up right and you are hosting your database in FileMaker Server, you should be able to logon using your Active Directory username and password. Try it. Does it work?

NOTE 1: Have you activated Instant Web Publishing? Type http://[name or IP address of FIleMaker Server]/fmi/iwp/ into your Internet browser and click the database link to open it.

NOTE 2: There is one idiosyncracy of Mac OS X and Windows OS. FileMaker Server running on OS X always authenticates users by looking to its local accounts first before looking on the domain if there is one. Windows is the opposite. This can cause problems where the user has an identical username for an account on the local machine and on the domain. To ensure user authentication occurs through the domain first, you should type in the Account Name the domain name followed by a forward slash (\) and the username. This forces FileMaker Server to look at the domain first. If you do this, Get(AccountName) will show the full username (including domain). So use PatternCount() to determine the actual username.