Saving The Browser
9/12/2003 3:31:01 PM
Some months back I became aware of the patent US 5,838,906 and the Eolas lawsuit against Microsoft, and followed a bit of conversation on the Net related to it. As many, I believed the issue would quickly go away because of ample prior art. Regrettably, this seems not to be the case.
It now seems that perhaps the browser itself and the browsing experience may have to be nontrivially modified as a result of the judgment. Although a bit late, if some of us perhaps dust off our old code, is there a chance that we could still save the browser through demonstration of clear prior art?
For my own interest, and for the record, I recently spent a little time pursuing my intuition that Lotus Notes R3 might be viable prior art relative to the patent in question. I am not an attorney, and I am surely not well versed in the nuances of the case, but it seems to me after initial investigation that there is indeed quite a bit of relevance.
I pursued this with the assistance of my brother, Jack Ozzie, and with another of my employees at Groove, Rob Slapikoff. Both Jack and Rob worked for me at Iris Associates in the development of Lotus Notes. Although I am personally responsible for a good deal of the “browser” code in question, I asked Jack to help because he specifically did all of the work related to our “object linking and embedding” technologies – first a Lotus technology referred to internally as DIP, (Document Interchange Protocol), and later in loose collaboration with the Windows & Excel teams on what was referred to as either CDP (Compound Document Protocol) or OLE (Object Linking and Embedding). I asked Rob to help because he was, in essence, a Lotus Notes “solution developer” at the time, and was very familiar with how one would quickly weave together a solution involving multiple applications.
When I began this investigation, I thought that it might be challenging to recreate a scenario, given the feature set available Notes R3, that was close to what was described in the patent. In fact, however, the hard part was only in putting together a computing environment that ran Notes R3. Once we had Notes running, it only took about 15 minutes to reproduce what I’ve shown below, and there was no programming involved. Meaning, everything done was done with just the out-of-the-box UI of both Notes and Excel.
First, let me describe the environment that we recreated. Since the filing date on the patent is October 17, 1994, I sought to obtain software that was clearly shipping to end-users before that date. I set about to assemble the following software to assist in the demonstration: Microsoft MS-DOS 6.22, Microsoft Windows for Workgroups 3.11, Microsoft Excel 5.0, and Lotus Notes 3.0. In my personal archives, I happened to be in possession of DOS, Windows, and a freshly shrink-wrapped copy of Notes. I selected Microsoft Excel 5 because information on the Web indicated that it shipped 12/14/93, and I easily obtained a shrink-wrapped copy via eBay in a matter of days.
I first used VMware Workstation 4 to create a virtual machine environment roughly comparable to that of the era. Then, I installed MS-DOS 6.22 within that virtual machine, as well as Windows for Workgroups 3.11. Finally, I installed Excel 5.0 and Notes 3.0. I chose WFW because I felt it to be very important to create a configuration that could be used as a “client/server” network environment between multiple virtual machines. As such, I installed both the Notes 3.0 client and server programs, and set about to creating the demonstration herein.
For those of you not familiar with Lotus Notes, but let me give you a very brief overview. I founded a company, Iris Associates, in December of 1984 to create the product eventually purchased and marked by Lotus Development Corporation as Lotus Notes. Notes first shipped in December of 1989, and at the time of this writing is in use by more than 100 million individuals at major corporations worldwide.
The product was and is an inherently “networked” client/server product. It consists of a client piece that is used to browse and create text, hypertext, and hypermedia documents, and collections of such that are woven into “applications” and “solutions”. These documents, applications and solutions are hosted on a server analogous to today’s “Web application servers”. Notes documents are, in essence, compound documents analogous to today’s “HTML documents”. They structurally consist of a stream of “CD Records” (Compound Document Records) which are, in essence, the logical equivalent in an HTML document of HTML “elements” or tags. The very name “Compound Document” chosen for such CD Records was chosen in the late 80’s during Notes development, and is indicative of how we envisioned documents as being compound combinations of different media elements. The format of CD Records was documented in the Notes developer toolkits.
The Lotus Notes client, which herein I also refer to as the “browser”, could use a number of methods to fetch documents from (or store documents to) the Notes Server. One way was through “document links”, which are analogous to the Web’s URLs, being “pointers” to documents. The general process used by the Notes browser to fetch and display a document was quite similar to that used by a Web browser: the Notes client would parse the link to figure out what server it needed to fetch the document from, it would then send a network transaction to that server, then the server would read the document from its internal storage, and respond back to the client with the contents of the document. Then, the Notes browser would begin sequentially parsing the series of records contained within, rendering the contents on-screen in a window as the records were being parsed.
Through use of “document links” that are used to create webs of interlinked Notes documents, Notes is naturally a hypertext system. In fact, in creating Lotus Notes I was quite influenced by the writings of Ted Nelson, about 10 years prior to starting work on it. In reading the patent, one of the more interesting questions pertained to embedded links to hypermedia documents. In fact, one of the “big deals” about the release of Lotus Notes R3 was its support of embedded media, and the inclusion of linking technology to enable networked hypermedia. To drive this point home, note that two diskettes were included in the Lotus Notes package, shrink wrapped along with the Lotus Notes diskettes themselves. One is entitled “Lotus Multimedia Tools” and the other is “Lotus Media Clips” with sample content.
Directory of Lotus Multimedia Tools [.] [..] ANGELSNG.WA_ ANNDLL.DL_ ANNICON8.DL_ ANNICONE.DL_ ANNICONV.DL_ ANNOTATE.EX_ ANNOTATE.HL_ ANNOTATE.SMI COMMDLG.DL_ INSTALL.EXE LMPDLL.DL_ LMPHNDLR.DL_ LOTUSSND.EX_ LOTUSSND.HL_ LSM.DL_ LTSANNOT.BM_ LTSANNOT.MAC LTSANNOT.SM_ LTSICN02.DL_ LTSMEDIA.BM_ LTSMEDIA.MAC LTSMEDIA.SM_ LTSSOUND.BM_ LTSSOUND.MAC LTSSOUND.SM_ LZEXPAND.DLL MEDMAN.EX_ MEDMAN.HL_ OLECLI.DL_ OLESVR.DL_ SHELL.DL_ VER.DLL WINHELP.EX_ WINHELP.HL_ 34 File(s) 1,079,263 bytes Directory of Lotus Media Clips [.] [..] BASSOON.WA_ BELL.WA_ CHARGE.WA_ COOLJAZZ.WA_ DEATHMAR.WAV DRUMROLL.WA_ FANFARE1.WA_ FANFARE2.WA_ FANFARE3.WA_ FANFARE4.WA_ FLUTEDUO.WA_ HOORAY.LS_ INSTALL.EXE JAZZY.WA_ LZEXPAND.DLL MELODRAM.WA_ OFF2RACE.WA_ PIZZICAT.WA_ RASPBERY.WA_ ROCKGUI.WAV SILLYSNG.WA_ STEELDRM.WA_ SUSPCHRD.WA_ VER.DLL WHEELO.LS_ WHOOSH.WA_ 26 File(s) 1,158,326 bytes
These were programs written by separate groups at Lotus that were “bundled” with the Lotus Notes application. As you can see by the content of these diskettes, Lotus’ concept was that documents could be “annotated” in a live fashion with linked-to media.
Note also that these programs used the Microsoft OLE libraries as their linking & embedding technologies, like Microsoft Excel itself. This is because at the time this was the easiest way to write programs using Microsoft OLE.
That said, Lotus Notes itself did not utilize the Microsoft OLE libraries, because the DDE-based linking and embedding support in Lotus Notes was developed prior to the emergence of OLE (as Lotus’ own DIP protocol). When OLE came along, developers at my company worked with developers at Microsoft (in particular, Ed Fries of the Excel team) to ensure the compatibility and interoperability of Lotus Notes and such OLE-embedded programs, via standardization on DDE-based inter-application protocols. When OLE was first publicly introduced to the world, we were on stage with Microsoft demonstrating such interoperability.
In 1993 or thereabouts, we saw the emergence of TCP/IP, HTML, HTTP, Mosaic and the Web. From our perspective, all of these were simplistic emulations of a tiny subset of what we’d been doing in Notes for years. TCP/IP instead of Netbeui or IPX/SPX, HTML instead of CD records, HTTP instead of the Notes client/server protocols, httpd instead of a Notes server. And we were many years ahead in other ways: embedded compound objects, security, composition of documents as opposed to just “browsing” them, and a sophisticated development environment. I am quite embarassed to say that we frankly didn’t “get” what was so innovative about this newfangled “Web” thing, given the capabilities of what had already been built.
The following series of screen snapshots (of the VMWare emulator window) walks you through a simple demonstration of Notes R3’s capabilities.
01 Windows Splash Screen
This just shows the general operating environment of this demo, and the rough timeframe in which Windows for Workgroups was prevalent.
02 Notes Program Group
This is to show what particular components I installed. This is the Lotus Notes for Windows Release 3. Note that the Lotus Notes client and server programs were also available for OS/2 in Release 3.0. Some months later, in a later maintenance release of Release 3, we also shipped the product for Windows NT (client and server), for the Apple Macintosh (client only), and for various flavors of Unix.
03 Office Program Group
This shows what components of Excel 5.0 were installed.
04 Notes Server Console
This shows the Lotus Notes R3 for Windows Server console while it is running. It is clearly using NETBIOS as the network transport protocol. This shows one remote user (Ray Ozzie/Groove) connected into the server in order to perform network transactions such as fetching documents.
05 Notes Browser Window
This is the Lotus Notes client user interface, maximized to full-screen. The “Workspace” icon in the center, when maximized, is a container where the user can remember “bookmarks” to commonly-used documents and applications. Applications in Notes are referred to as “databases”, and are analogous to Web Sites in that they are collections of documents and application logic brought together for a specific purpose.
06 Notes Splash Screen
This is the Help About display, showing the precise version of Lotus Notes in use for this demonstration.
07 About to click Hypertext Link
In this example, I have first resized the Notes browser window to only be a part of the screen, for reasons you’ll see later.
Then, I used a Notes Workspace “bookmark” to fetch and open a document called “Start Here”, which is a Notes document (consisting of CD records) fetched over the network from the Notes Server. Such a document is analogous to a Web Page, and it’s being browsed here in the Lotus Notes client. This document has some text, and has two inter-document “links” that could potentially be pointing to other documents on this server, or on other servers.
Now, I’m going to double-click the first link (the one that’s highlighted), which will open the linked-to document into a new browser window…
08 After clicking Hypertext Link
This is what things look like immediately after I double-clicked that first link. The Notes browser opened another window on the screen, and the linked-to document was fetched, and displayed as it was parsed.
Now, I close this window and return to the original document…
09 About to click Hypermedia Link
Now, as you can see, I immediately double-click this second link, which again opens a new document into a new window. But in this case, the linked-to document is a document with an embedded hypermedia object reference, and you’re about to see what happens.
In essence, the linked-to document we’re about to open has embedded CD records that describe 1) that an object is to be embedded into the page, 2) the class of object (in this case “EXCEL”), 3) the pointer to the object (in this case X:\TEST.XLS”, which is a reference to a spreadsheet object that is potentially on a remote server computer represented by drive letter X., and 4) a parameter indicating whether or not this object should be immediately activated upon browsing, or whether the object needs to be double-clicked before activation.
In this example, we’ve set up the object reference on the linked-to page so that it does automatic activation, so quite a bit will be happening when we just simply double-click the link in the above document. Here goes…
10 After clicking Hypermedia Link
Everything that appears on the screen above appeared automatically as a result of simply opening the linked-to hypermedia document. Meaning, when I took the screen shapshot above, I hadn’t clicked or typed anything beyond the double-click to follow the link.
If you look at the “Hypermedia Document” window, you’ll first see some text that was rendered, and then you’ll see a rectangle displaying the spreadsheet, then you’ll see two buttons. The rectangle is an up-to-date rendering of the “live” current state of an external object, as you’ll see in a moment. Note that as the internal CD object tag was parsed by the Notes browser, the external Excel application was launched, and the referenced spreadsheet was activated into it. Then, an active inter-connection between the Notes browser client and the external application was initiated, and the rendering of the object was sent from the external application to the browser client. This inter-application connection remains intact. All communication between applications is done using protocols riding on the DDE Windows messaging transport.
To demonstrate the “live” nature of the interconnection, I’ve previously placed two buttons on the spreadsheet itself, which assign the value of cell B2 to a fixed, known value. Similarly, I’ve previously placed two buttons within the Hypermedia Document that communicate with the active, linked-to application, telling it also to set the value of cell B2 to a fixed known value.
11 After clicking Set 111 button in Excel
So, as I click the “Set 111” button, note that it sets the cell B2 to 111. Note that not only is the spreadsheet updated, but also the active rendering of that spreadsheet object is updated in the Hypermedia Document. Thus, you can see that the two applications are actively inter-connected.
As you watch this demonstration, it’s important to note that this version of Excel was the first to have ODBC support. Thus, the cells being displayed, or perhaps a chart, could be the result of a remote query to a very large SQL database. Furthermore, in this example, setting the value of a cell could just as well have been the “trigger” to a series of very complex commands within the spreadsheet itself – meaning that the application could have been far more robust than just setting the value of a cell.
12 After clicking Set 222 button in Excel
Again, just to prove the point that you could have multiple “controls” in the spreadsheet viewer itself, the second button sets the value to “222”.
13 After clicking Set 333 button in Browser
Now, for something completely different, I click the button within the Hypermedia Document called “Set 333”. This actually uses the active inter-application connection via DDE to, in essence, “drive” the separate application. In this case, it’s setting the value of that same cell to 333. Again, remember that this might’ve instead caused a fetch of a small piece of data from a very large remote data set.
And again, note that not only is the spreadsheet updated, but also the active rendering of that spreadsheet object is updated in the Hypermedia Document. Thus, you can see that the two applications are actively inter-connected.
14 After clicking Set 444 button in Browser
And finally, I just set it to 444 to show that you can have multiple commands in the browser document that do different things
Now, let me take a very brief look at the claims in the patent, and quickly review how I believe this relates to this particular prior art.
Notes has a client workstation and a network server, coupled by the network, and this is the distributed hypermedia environment. The Notes client is the browser application that parses the distributed hypermedia Notes document to identify text formats, setting up links and buttons, etc. When opened, at least a portion of a hypermedia document is displayed on the client, which has contained within it a compound document record indicating that an object is embedded. The object reference has type information associated with it that enables it to identify the executable application, and the browser can automatically invoke the application to execute on the client to display and enable interaction with the object. The display area within the document is processed interactively, dynamically and interactively updating when controls are manipulated either within the browser application or within the executable application itself.
Indeed, the executable application is controllable and can be controlled via DDE-based inter-process communication between the Notes browser and the external application.
Again, for the entire duration that the browser window is being displayed, there is active communications between the browser application and the controllable external application.
I didn’t demonstrate this, but imagine simply that we had enhanced the demonstration such that the Excel spreadsheet had used ODBC to issue queries to a very large data set on the network server running a SQL server. In this case, one could press a button on the client workstation (in either the browser or Excel), the query would be execute on the network server, and would send the result set back. The client application would then respond to the data and would have functions or macros that would behave differently based upon the data returned from the server. Thus, the client workstation would be using the retrieved information to further control the client application.
Finally, claims 6-10 are identical to claims 1-5, with the substitution of “The computer program product” instead of “The method”. Well, yes, we did create an actual product to do such a thing. We even shipped it about 18 months before his filing. Lotus was a public company and at the time one of the biggest forces in the personal computing industry, so surely the person or persons doing the patent filing must have or should have known about our hypermedia innovations. Given all the press coverage, he was likely also influenced by them in envisioning his own distributed hypermedia enhancements to the then-nascent Web browser technology.