MXMLiszt version 0.9.0 is now available for download.

MXMLiszt is a web-based delivery and search/retrieval environment for MusicXML files and their manifestations.

MXMLiszt was created in order to complete a Master’s in Library and Information Science at the University of Alabama under the direction of Dr. Steven L. MacCall.

The documentation and source-code download links are available here.

The accompanying research paper, “Beyond Images: Encoding Music for Access and Retrieval” can be accessed here.

As of June, 2010 the live demo of MXMLiszt can be accessed at:

http://opensourcelibrarian.org/MXMLiszt

MXMLiszt is licensed under the BSD software license.

MXMLiszt 0.9.0 (beta)

---

TERMS:

MXMLiszt is licensed under the BSD software license.

MXMLiszt

Copyright (c) 2010, Nitin Arora.

All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

* Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

* Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

* Neither the name of Nitin Arora nor the names of his contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

---

Table of Contents

Introduction

Installation

List of primary files and folders

How it works/Getting Started

Troubleshooting

FAQ

Future Work

Live Demo

Acknowledgments

---

Introduction

MXMLiszt is a web-based delivery and search/retrieval environment for MusicXML files and their manifestations.

MXMLiszt is currently known to work on the following 32-bit Windows servers.

Successfully tested for production and hosting on Windows XP Home Edition, Service Pack 3.

Successfully tested for production and hosting on Windows 7, Professional.

Successfully tested for hosting on Windows Server 2003.

MXMLiszt 0.9.0 was written by Nitin Arora.

MXMLiszt was created in order to complete a Master's in Library and Information Science at the University of Alabama under the direction of Dr. Steven L. MacCall.

The accompanying research paper, “Beyond Images: Encoding Music for Access and Retrieval” can be accessed at http://blog.humaneguitarist.org/uploads/MXMLiszt/Beyond_Images__Encoding_Music_for_Access_and_Retrieval.pdf.

MXMLiszt uses the following open-source applications:

Musescore: version 0.9.5

MusicXML Library: version 2.0

ImageMagick: version 6.6.1-5

Xenoage Player: version 0.4.1*

Saxon-HE: version 9.2

BaseX: version 6

*As of 2010-02, the Xenoage Player is no longer supported. However, the newest version, Zong!, is still in an early alpha stage, hence the decision to use the Xenoage Player in conjunction with MXMLiszt. Future versions of MXMLiszt will hopefully incorporate both the Zong! Player as well as the developmental MusicXML viewer, Zong! Viewer.

On the name "MXMLiszt":

MXMLiszt = MusicXML List.

MXMLiszt is pronounced [mak-suh-muh-list].

MXMLiszt references the great musical maximalist, Franz Liszt.

---

Installation (Windows 32-bit, home-based server)

You need an Apache server that supports PHP and Java. You probably have the Java Runtime Environment already installed; if not, you can find it here.

1. If you don't have an Apache/PHP setup, XAMPP Lite is an excellent choice to get started:

   • Extract xampplite-win32-1.7.3.zip.

   • Unzip the file to wherever you like. I like to rename the root XAMPP folder to “www”.

   • Run setup_xampp.bat and setup your server.

   • For security reasons you might want to delete the phpMyAdmin and mysql folders and all files/folders in htdocs. You assume all risks regarding security, etc.

   • Create a "bin" folder inside the root XAMPP folder.

   • Because some of the scripts take a lot of time to run, increase the maximum execute time in php.ini to more than the default of 60 seconds. I used “3600”.

2. Download the MXMLiszt-0.9.0.zip file.

   • Extract the files/folders and place the root MXMLiszt folder inside XAMPP's “htdocs” folder.

   • Open adminPanel.php and change line 14 (below) to replace the default Administrator password from “tausig” to your own password.

     • $validPassword = "tausig"

3. Download the Xenoage player from http://www.xenoage.com/downloads/xenoplay/xenoplay-0-4-1-src.zip.

   • Extract the two .jar files into the htdocs/MXMLiszt folder.

4. Download MuseScore from http://sourceforge.net/projects/mscore/files/mscore/mscore-0.9.5/MuseScore-0.9.5.exe/download.

• Install MuseScore inside the “bin” folder. Rename the folder to simply “MuseScore”.

  • The program executable is now located at bin/MuseScore/bin/mscore.exe

• Run mscore.exe and go to Edit>Preferences>Export and uncheck “Screen shot function”.

• Hit “Apply” and close the program.

5. Download the MusicXML Library from http://sourceforge.net/projects/libmusicxml/files/libmusicxml-win32/libmusicxml-2.00-win32/libmusicxml-2.00-win32.zip/download.

• Create a folder called “libmusicxml” inside your “bin” folder and place all the MusicXML Library files in that folder as such:

6. Download ImageMagick from http://www.imagemagick.org/download/binaries/ImageMagick-6.6.1-5-Q16-windows-dll.exe.

• Install ImageMagick inside the “bin” folder. Rename the folder to simply “ImageMagick”.

  • The “convert” executable is now located at bin/ImageMagick/convert.exe

7. Download BaseX6.jar from http://sourceforge.net/projects/basex/files/basex/6.0/BaseX6.jar/download.

• Place it in the “bin” folder.

8. Download saxonhe9-2-1-1j.zip from http://sourceforge.net/projects/saxon/files/Saxon-HE/9.2/saxonhe9-2-1-1j.zip/download.

• Extract the saxon9he.jar file to the “bin” folder.

9. To get things started, run “xampp-control.exe” from the root XAMPP folder and start only the Apache module:

You should now be able to go to “http://localhost/MXMLiszt/” from your browser and see the MXMLiszt welcome page:

---

List of primary files and folders

File	Type	Path (relative to MXMLiszt root)	Description
style.css	style sheet	.	controls the HTML display
adminMenu.html	module	.	lists administrator scripts to be executed
documentation.html	document	./documentation	MXMLiszt documentation
footer.html	module	.	contains closing "body" and "html" tags
index.html	document	.	framed view of MXMLiszt; standardizes view across various display resolutions
menu.html	module	.	navigation menu
adminLogin.php	script	.	requests password for administrator access
adminLogo.php	module	.	displays image of Franz Liszt if administrator is logged on
adminPanel.php	script	.	displays "adminMenu.html" if administrator is logged on; note: editing the value of the "$validPassword" variable will change the administrator password. The default password is “tausig”.
concatMODS.php	script	.	concatenates all files in "/mods"; allows XQuery over one file rather then several separate files
concatMXML.php	script	.	concatenates all files in "/musicXML"; allows XQuery over one file rather then several separate files
displayMODS.php	module	.	displays MODS in browser per the" mods.xls" stylesheet
galleryMXML.php	script	.	allows user to browse images of the first page of each piece in "/musicXML"
generateGallery.php	module	.	generates HTML code to displays preview of the first page of each piece in "/musicXML"
generateIndex.php	module	.	generates HTML code to displays index view of each piece in "/musicXML"
generateMODS.php	script	.	generates preliminary MODS metadata using the Saxon XSLT processor and the "mxml2mods.xsl" XSL transformation sheet
generatePDF.php	script	.	calls Musescore to generate PDFs for all files in "/musicXML"
generatePNG.php	script	.	calls Musescore to generate PNGs for all files in "/musicXML"; deletes all PNGs for pages > page 1; calls ImageMagick to resize the image
indexMXML.php	script	.	allows user to browse list of each piece in "/musicXML"
iniSet.php	module	.	sets maximum execution time for time-intensive scripts like "generatePNG.php"; default is 1 hour
inputMIR.php	script	.	allows user to send XQueries to BaseX via the web
inputXQ.php	script	.	allows user to send XQueries to BaseX via the web
loadMODSasDC.php	module	.	filters down MODS names/titles to Dublin Core creator/title for index and gallery views
normalizeMXML.php	script	.	calls Musescore to open and close all files in "/musicXML" to theoretically normalize the MusicXML output
outputMIR.php	script	.	displays XQuery results from "inputMIR.php" entered via the "Faust" terminal
outputXQ.area.php	script	.	displays XQuery results from "inputXQ.php" if user used the "Dante" terminal
outputXQ.mods.php	script	.	displays XQuery results from "inputXQ.php" if user used the query box
processXQ.php	module	.	writes queries to temp file and writes BaseX results to another temp file
purgeTEMP.php	script	.	administrator script to delete all files in "/temp" directory
reportMODS.php	script	.	separates remediated and unremediated MODS files into two lists; unremediated MODS files have as the last line the commented-out phrase: "<!–Preliminary MusicXML to MODS metadata created via XSLT–>", metadata librarians must delete that last line when they have remediated the MODS metadata for the file to be considered remediated
setup.php	module	.	contains the paths and arguments to the open-source applications that MXMLiszt relies on. Changing these values is necessary only if you use a different folder structure than prescribed in the installation instructions.
showAccess.php	script	.	displays Apache access.log file; purges contents of access.log
transmuteMXML.php	script	.	allows user to access manifestations of MusicXML files
transposeXML.php	script	.	calls MusicXML Library to transpose MusicXML file; allows user to access manifestations of transposed MusicXML files
welcome.php	module	.	home page of MXMLiszt
sampleMIR.txt	document	.	contains some Music Information Retrieval queries that the user can paste into "inputMIR.php"
mods.xsl	XSL	.	displays MODS metadata files as HTML
mxml2mods.xsl	XSLT	.	determines what elements in MusicXML to map to MODS metadata
concat	folder	.	contains concatenated MODS and MusicXML files
images	folder	.	contains images needed by the website for pleasant display
mods	folder	.	contains all MODS metadata
musicXML	folder	.	contains all MusicXML files; all files must be in root folder
pdf	folder	.	contains all PDF files
png	folder	.	contains all PNG preview files
temp	folder	.	contains all temporary files
BaseX6.jar	executable	..\..\bin\	XQuery processor
saxon9he.jar	executable	..\..\bin\	XSLT processor
skin.jar	skin	.	skin files for Xenoage Player; this applet skin must be placed in "htdocs/MXMliszt"
xenoplay.jar	executable	.	Xenoage MusicXML audio player; this applet must be placed in "htdocs/MXMliszt"
bin	folder	..\..\	contains ImageMagick, MusicXML Library, Musescore, BaseX, and Saxon.
ImageMagick	folder	..\..\bin\	contains ImageMagick files\folders
libmusicxml	folder	..\..\bin\	contains MusixXML Library files\folders
MuseScore	folder	..\..\bin\	contains MuseScore files\folders
convert	executable	..\..\bin\ImageMagick\	resizes PNG files
xmltranspose	executable	..\..\bin\libmusicxml\	transposes MusicXML files
mscore	executable	..\..\bin\MuseScore\bin\	normalizes MusicXML files; generates PDF and PNG files; can generate sound files as well (not implemented in MXMLiszt 0.9.0)

 

---

How it works/Getting Started

Screencasts coming soon – these will hopefully be the best and easiest way to see how MXMLiszt works.

MXMLiszt works on the principle that while the MusicXML encoding for a given composition is itself a manifestation of a work (i.e. the “composition”), the MusicXML file itself can function as a sub-work capable of birthing derivative image and audio files; manifestations also include musical transpositions in MusicXML format, thus allowing the the transpositions to serve as sub-sub-works also capable of spawning new image and audio content.

The following graphic attempts to depict the essential overall framework of the platform.

Initial Setup

It is by first running administrative PHP scripts that the MXMLiszt collection and search environment is created for one or many pre-existing MusicXML files on the server.

By describing the completion of an administrative "cycle" for a MusicXML file called "foo.xml", it is simpler to demonstrate via example the process depicted in the diagram above.

Administrative scripts available are as follows:

Setup Scripts

• Normalize MusicXML

• Generate MODS

• Generate PDF

• Generate PNG

Search Related Scripts

• Concatenate MODS

• Concatenate MusicXML

Reporting Script

• MODS remediation list

a. Setup Scripts

Once MusicXML files are placed on the server's “MXMLiszt/musicXML” folder, setup scripts prepare the platform so that a user may browse the collection and have access to images and audio of the compositions.

1. Normalize MusicXML

Variations occurs between MusicXML files exported from different notation software packages. Due to this variation, it seems necessary to "normalize" each MusicXML. That is to say, it seems necessary to import each MusicXML file – irrespective of which notation software it was generated from – and export it from the same application, namely MuseScore. This helps to standardize the MusicXML files, allowing for more consistency in regard to subsequent image creation by MuseScore and metadata creation via XSLT.

In a real working environment our test file "foo.xml" would now have been opened and re-saved by MuseScore.

MXMLiszt's "index view" option would now result in displaying the following:

Prematurely clicking on the link above would be less than ideal in that no descriptive metadata is yet available. Also, since no PDF is readily available one would have be be created "on the fly", slowing down the user experience:

Nevertheless, preliminary functionality exists at this point allowing the user to see and hear the original composition as well as access transpositions of the piece.

2. Generate PDF

Though scripts A2 through A4 may be run in any sequence, it is best to generate PDFs first otherwise PDFs will be generated by user activity, adding to the amount of time a user has to wait before being able to view the PDF file.

This script simply would instruct MuseScore to open "foo.xml" and render a file called "foo.pdf" in the “pdf” folder. If this PDF already exists, no new file will get rendered.

It should be notated that this script can be fairly time intensive for large amounts of MusicXML files in the collection.

3. Generate PNG

Though not necessary and strictly an added "feature" of the platform, this script would call MuseScore to output a PNG image file of each page of each MusicXML file in the “png” folder. It would then delete all but the first page. The ImageMagick software is then called to resize all remaining PNG files to a near thumbnail image, leaving a smaller preview image, "foo.pre.png". If this PNG file already exists, no new PNG files will get rendered for “foo.xml”.

This preview image can be viewed by hovering over a selection in the "index view" of the platform:

or by using the "gallery view" of the platform (i.e. visual browsing):

The usefulness of this feature is more evident in cases where the online collection contains more than one MusicXML file:

The rational behind this feature was to emulate some print editions of sets of musical compositions in which the user is provided a "visual table of contents", a page containing the first measure or so of each composition within the edition, the "incipient", so that traits such as primary key signature and music texture can be gleaned for each composition within the edition without having to flip through the entire manuscript.

Given the additional processing, this script is naturally even more time consuming than the previous PDF script.

4. Generate MODS

As stated prior, it is via XSLT that one is able to automatically crosswalk descriptive metadata elements in MusicXML to MODS while remaining in the domain of XML technologies, as opposed to using a scripting language such as PHP, Python, etc. to achieve the same results.

The XSL transformation sheet used by the platform is called "mxml2mods.xsl" and seeks to create MODS XML files for the following primary descriptive properties about the composition:

titles (primary, alternate)

creators (composer, lyricist, arranger)

instrumentation

Additional elements include the unique, such as the URI identifier, and the fairly generic, such as the Library of Congress Subject Heading "sheet music". The encoding date of the normalized MusicXML file is also captured.

Running the Generate MODS script would create a file entitled "foo.mods.xml"

The "index view" of the platform would now show descriptive metadata as such:

The Gallery view of the of the platform would show the metadata encapsulated in a top-centered yellow box only when the user hovers over the preview image of the composition:

Note that the metadata, though MODS, is showing up on these pages as Simple Dublin Core, with the "DC" prefix. This was done for two reasons: 1) Ease of programming, given that the platform exists solely for the purpose of demonstration to the library community and 2) because it seemed simpler from a mere browse perspective to have the metadata be fairly "filtered down" to Dublin Core.

Clicking on the selection from either the Index or Gallery views will, however, now reveal the MODS record as shown below:

MXMLiszt uses an XSL stylesheet to display the MODS record in a user-friendly way as above, though librarians and information specialists can open the MODS record in a separate browser tab and view the page's source code to view the full MODS XML record.

At this point, everything is ready from a user browse perspective as they will have access to MusicXML, descriptive metadata, and image files. The Xenoage player will produce sound in real-time and thus no sound files need to be made.

b. Search Related Scripts

The two scripts related to preparing the platform for search/retrieval and Music Information Retrieval (MIR) can be run in any order.

1. Concatenate MODS

In order to create an adequate search environment for the descriptive metadata (MODS) on the platform, a "virtual" database file for all descriptive metadata is created via this script. In other words, the data from all MODS files are concatenated into one "super" file as such:

<hyperMODS>

<hypoMODS file="foo.xml">

1st MODS document

</hypoMODS>

<hypoMODS file="foo2.xml">

2nd MODS document

</hypoMODS>

…

<hyperMODS>

It is this hyperMODS file that is access via XQuery searching and the BaseX XQuery processor. This allows for the processor to access only one XML file as opposed to iterating through all individual MODS files on the platform, as the latter option would yield unbearable long retrieval times.

2. Concatenate MXML

Similarly, this script concatenates the data for all MusicXML files on the server for quicker search and retrieval in regard to MIR functionality. Given how much larger MusicXML files are in contrast to MODS files, this concatenated "super" MusicXML file can be extremely large, depending on the amount of MusicXML files in the online collection.

Note the similar structure to the hyperMODS file above:

<hyperMXML>

<hypoMXML file="foo.xml">

1st MusicXML document

</hypoMXML>

<hypoMXML file="foo2.xml">

2nd MusicXML document

</hypoMXML>

…

</hyperMXML>

By using administrative scripts to prepare these large concatenated XML files the platform is initialized for search/retrieval and MIR across the entire online collection.

The descriptive metadata can be searched by a user-friendly drop-down box or by manual entry of valid XQuery syntax into a terminal style input box – nicknamed "Dante" for one of Liszt's symphonic works. In fact, MIR searches can also be initiated via terminal input, however this terminal is fairly restrictive in that query results must return only MusicXML filenames so that the platform can display the results in either the Index or Gallery view styles.

A less restrictive terminal ("Faust", also for one of Liszt's symphonies) allows advanced users to perform MIR (and even descriptive metadata) searches and return results in the output style of their choosing. Currently, intermediate users can access a link to a plain text file with pre-written MIR queries that can be cut and paste into the terminal.

Obviously, both terminal input boxes are primarily intended for advanced users who understand both XQuery fundamentals and the file and folder structure of the MXMLiszt platform. Ideally future versions of the platform would support graphics-based input of musical information, such as pitches and durations.

c. Reporting Script

As mentioned, the XLS transformation that produces initial MODS records from the the MusicXML files ends the outputted MODS file with a commented-out line that reads:

<!–Preliminary MusicXML to MODS metadata created via XSLT–>

The "MODS remediation list" script simply reports to interested administrators (i.e. metadata librarians) which MODS files within the platform do and do not contain this line, the principle being that remediation (improvement) of MODS records by librarians would include the removal of this line, hence allowing subsequent remediated MODS records to be reported as remediated.

---

Troubleshooting

 

Administrator errors:

1. generateMODS.php not working

• Make sure Java is installed.

• You must have a live internet connexion for Saxon to perform XSLT operations due to xsd/schema needs.

 

User errors:

1. Xenoage Player not working

• Make sure Java is installed.

• Make sure you've installed the Java Soundbank.

• Occassionally, there are MusicXML files Xenoage cannot play. I plan to contact the programmers and ask why this might happen from time to time.

2. Some of the PDFs have very odd spacing and page layout issues.

• I think this is an issue with MuseScore. I own Finale Notepad and it seems to do a better job with MusicXML import. But, I really don't know yet why these things happen from time to time with certain pieces.

---

FAQ

1. Why are you using the Xenoage player? MuseScore can create OGG files from free SoundFonts and the audio can be embedded in the HTML 5 <audio> tag.

• I'm not clear on the licensing situation with SoundFonts even though there are many "free" ones available. Also, Xenoage doesn't require OGG files to be written and saved to the server, so the Xenoage applet saves storage space as well.

• I have thought about adding an OGG/<audio> tag option as a backup for MusicXML files that Xenoage is not able to play.

---

Future Work

• As of 2010-07, a permanent URL to the manifestations of "foo.xml" would look like this: /MXMLiszt/transmuteMXML.php?fname=musicXML%2Ffoo.xml.
  • I'd like to make these prettier, for example: /MXMLiszt/foo/
  • This tutorial on URL redirecting might have the answers I need in order to accomplish that.
  • This isn't really a programming issue, but more a server management issue. Still, it's better digital library practice to have tinier URLs.

---

Live Demo

As of June, 2010 the live demo of MXMLiszt can be accessed at:

http://opensourcelibrarian.org/MXMLiszt

---

Acknowledgments

Thanks to the following people:

Dr. Steven L. MacCall,

Jody DeRidder,

Tonio Loewald,

“Playingwithbrushes” for the great background used by MXMLiszt,

everyone responsible for the open-source applications and scripting languages used by MXMLiszt,

everyone responsible for all the great SMR/MIR tools, research, and resources on the Web,

Michael Good for MusicXML,

and Franz “The Man” Liszt.

Yesterday I wrote about trying out a Windows VPS using KickAssVPS.com to run MXMLiszt, my web demo for my music encoding research project.

Well, so far so good. Sort of …

I use Windows XP Home Edition (SP3, 32-bit) on my laptop, so my software and everything that it utilizes runs well on XP since I naturally made sure everything works.

But to my surprise things didn’t go that great when I uploaded everything to my VPS which uses Windows Server 2003. Here’s what was problematic:

1. Musecore version 0.9.5 is what I call to make PDFs and PNG preview files from the MusicXML files. On the VPS, the titles of the pieces weren’t showing up when these image files were getting made, so I went on to version 0.9.6 which is still in beta. It’s working OK. The PDFs are getting made when one transposes a piece, but they look better with the XP version of MXMLiszt on my laptop which uses version 0.9.5 of MuseScore. But, there’s something funny going on with the administrative PHP script that makes PNG preview files for the site. These are small image of the 1st page of each piece so the user can get a feel for the piece, its key and initial texture, etc. Not all the previews are getting made. Weird.
2. I also use MuseScore to "normalize" each MusicXML file by opening/saving them in MuseScore so that every file gets "spit out" by the same application before I use an XSL transformation to auto-generate MODS metadata from the MusicXML files. Works great on XP and with MuseScore 0.9.5 but on the VPS with the beta version of MuseScore, the title of each piece is showing up in the MODS as the primary and sub-title – so it’s redundant. If you don’t know what I’m talking about, don’t worry. Bottom line is things are wonky.
3. The software that makes the transpositions, MusicXML Library version 2.0, wasn’t working on the VPS. So I went down to version 1.09. Moreover, my PHP script that calls the MusicXML Library to transpose the music per user request looks for the "xmltranspose.exe" executable in version 2. The older version of the software calls that file "transpose.exe". So rather than change my script, I just renamed "transpose.exe" to "xmltranspose.exe" on the VPS.

So in other words, MXMLiszt really isn’t too compatible with Windows Server 2003. It’s working well enough as a live demo now, but I wouldn’t try and run it "for real" on the VPS since the PNG preview and MODS generation are problematic. So the production version is the XP version as on my laptop.

Oh well. The user doesn’t miss out, it’s just the Administrator who has to do the work on a local machine and upload everything to the VPS for the sake of having a live demo.

Such is life.

p.s. it’ll be a few more weeks before I can post the link to the demo. It needs more documentation before I can share it with anyone other than my professor. Feel free to email me off-list if you would like me to send the link as long as you agree not to post the link until the documentation is done.

---

This blog post is part of a semester-long investigation into digital encoding of symbolic music representation (SMR), its context in libraries, web-based delivery, preservation and metadata, and search and retrieval technologies..

Before I provide a quick update on the status of the web demo for my MusicXML digital library delivery mock-up, it goes without saying that I haven’t been bloggin per my original intent. That is to say, the initial idea of writing my paper with a modular approach fed by blog posts, has essentially been scrapped. The idea of it all seemed a bit too piece-meal to me. Instead, I’ll write the paper and post it in its entirety when it’s been written.

As for the web demo, it’s more or less completed from a programming p.o.v at this point, but there are a few things I want to mention:

• The software has a name in MXMLiszt, i.e. "MusicXML List". It’s pronounced "maximalist" and happily references my favorite composer in the process.
• It should run "out of the box" (after you install all dependencies) on Windows servers running XAMPP or at least an Apache installation on a Windows server with the same folder configuration as XAMPP.
• It will utilize open-source software only, with the ironic exception of the OS.
• MXMLiszt will be released with a BSD license approximately 6-8 weeks from now after the documentation is complete and my paper is finished.

Eventually, I’ll port MXMLiszt to Linux (in all likelihood for Ubuntu specifically), but that will entail some work I’m not willing to be distracted by just yet. With Linux, I’ll have more homework to do before I can distribute it and ask people to expect it to work.

Other problems are finding a server to support MXMLiszt. Given all the software requirements and that I *do not* want to have to rearrange  – for either a Windows or Linux server – all the paths to the various executable I utilize and - for Linux hosting – the fact that I can’t control what Linux distro a web hosting company offers, I have only two options:

1. Use my own laptop as a server. The downside of this is that I can’t leave it on 24/7. I’ll only be able to offer it from roughly 9-5 EST Mon-Fri since I’ll need my computer to rest and/or be available for personal use while I’m home.
2. Run MXMLiszt from a Virtual Private Server (VPS) – i.e. instead of a typical web hosting plan, I’ll rent out a virtual machine running Windows from a hosting provider. This gives me total control of the VM and I don’t have to make compromises and/or rearrange folders and paths to executables, etc.

I’ve decided to go with the 2nd option for now using KickAssVPS. I’ll report back how that works out, but I want to close with the economic dilemma all this presents.

It’s roughly $35 a month to have a Windows VPS with KickAssVPS. And that’s on the low end of VPS pricing. That’s not that expensive but when one factors in that for that rate I could finance a brand-new really high end laptop, I have to ask myself "Why not just use my current low-end Dell laptop as a server and use the $35 bucks a month to get a really nice laptop?".

Good question. For now I’ll try the VPS, but I have a feeling that my narcissistic need to control everything will lead me back to running a server from home. And I think in the end I just won’t be able to justify the cost of renting (VPS) vs. renting to own (financing a new laptop).

If you’re asking "Why get an expensive laptop? Why not get a really cheap one or better yet a used headless server?"

Well, my friend, it doesn’t get much cheaper than my Dell Vostro. If I buy a low-end laptop, it’ll end up being comparable in price and specs to my Dell. It doesn’t make much sense to essentially have two of the same thing. As for headless servers, that would be better from the p.o.v. of running a server 24/7 but I’d need a monitor at some point and I don’t want to get into buying more peripherals …

… unless someone can tell me how to remotely control a headless server from a laptop without *ever* needing a monitor for the headless.

---

This blog post is part of a semester-long investigation into digital encoding of symbolic music representation (SMR), its context in libraries, web-based delivery, preservation and metadata, and search and retrieval technologies.

Just a quick morning post today …

The last 10 days or so I’ve been struggling with some major problems that arose in trying to implement effective XQuer-ies on my web demo.

1. Dublin Core doesn’t allow me to differentiate creator "types", so I was limited to searching across the DC:creator element for all creators, be they Composer, Lyricist, or Arranger. MusicXML does differentiate these types, so essentially Dublin Core was making me "dumb down" some information. I want people to be able to search creator specifically by their role: Composer, Lyricist, or Arranger.
2. I needed a way to iterate an XQuery over all the MusicXML documents and I needed it to be relatively fast. A demo is a demo, but impatience is impatience and I just can’t accept slow query processing.
3. The XQuery processor I was using didn’t support some XQuery functions that would allow a searcher to type in "Bach" and retrieve documents for which the creator was "J.S Bach", "Johann Sebastian Bach", "Bach, J.S", "P.D.Q Bach", etc. This really was limiting the search/query coolness factor and I wasn’t at all happy about it.

Here are my solutions (details to follow in a few days or so):

1. Ditch Dublin Core and switch to MODS, which does allow me to specify the role of a creator. Last week, I made a MusicXML to MODS XSL transformation for descriptive metadata and it’s working well.
2. Steal an idea from Using XQuery on MusicXML Databases for Musicological Analysis so that rather than iterate one query (say for the number of notes in a piece) across multiple MusicXML docs, I just concatenated all the MusicXML documents. The original files are left alone, but a "super" MusicXML file gets created so that one can just query that one file, hence no need for lengthy iteration. I’m not sure how those fellows did it, but I just automated it via PHP using the following format:

 <hyperMXML>

<hypoMXML file="foo1.xml">

1st MusicXML document

</hypoMXML>

<hypoMXML file="foo2.xml">

2nd MusicXML document

</hypoMXML>

</hyperMXML>

3. Switch XQuery processors! I’ll go into the ones that didn’t support the function I needed another time, but I will say that BaseX did the trick. Below is the query that searches for creators with "Bach" somewhere in MusicXML’s <creator> element. For the deliverable demo, I won’t be querying these big MusicXML documents for simple descriptive metadata like Creator, that’s what the MODS is for. But this is just an example. The "ftcontains" syntax is what allows for retrieval of values where "Bach" is somewhere within the element, but isn’t necessarily equivalent to the entire element value.

for $i in doc("../temp/concat/concatMXML.xml")/hyperMXML/hypoMXML/score-partwise
where $i/identification/creator ftcontains "Bach"
return ($i/work/work-title)

---

This blog post is part of a semester-long investigation into digital encoding of symbolic music representation (SMR), its context in libraries, web-based delivery, preservation and metadata, and search and retrieval technologies.

This is my final semester at the School of Library and Information Studies at the University of Alabama.

For my final credits, I’ll be researching the digital encoding of symbolic music representation (SMR), i.e. "sheet music", its background, and the benefits it offers over simply thinking of SMR as an image – all within the context of libraries, including avenues of web-based delivery, preservation and metadata, and search and retrieval technologies. My research will be directed by Dr. Steven MacCall.

In addition to a paper addressing these issues, I’m required to deliver what is essentially a "demo" of a MusicXML web-based delivery system that could serve to demonstrate to librarians the possibilities that arise with the usage of XML-encoded musical information.

I’ll be blogging along this semester as part of a modular approach to constructing the paper. Currently, I’m plugging away at the demo which utilizes open-source server-side music applications.

The first idea behind the demo is that one could "drop" MusicXML documents on their server and – via automation – PDF, audio, and preliminary Dublin Core metadata are generated. So far, I’ve got all that covered but I need to polish the output and I really need to comment-up my PHP code as I’m even forgetting at times what I’ve done and why. I’ll be sharing the code as well as the XSL transformations that are used to generate the Dublin Core metadata from the MusicXML documents.

As time allows, I’ll try and add some cool features. For example, this week I implemented libmusicxml so that the user can generate a PDF of a musical score in a different key than the original. I’m guessing this is the same way that the Wikifonia site offers this transposition option. I’ll have to ask them to make sure.

The second idea behind the demo is to implement a search/retrieval mechanism using XQuery. This will prove the biggest challenge as I don’t know XQuery well, but I have some excellent sources on querying MusicXML documents from which I can learn. Implementing XQuery on the site has – for the most part – already been dealt with in terms of scripting. In other words, the hard, hard work is done in terms of the demo, but I’m not quite there yet …

ps: Unfortunately, the server-side software required to run the demo can’t be added to a free server like the one I use for this blog. But while I can’t host the demo, I will at the end of the semester offer it as a download-able package so that one can run it on their personal computer as server.

---

This blog post is part of a semester-long investigation into digital encoding of symbolic music representation (SMR), its context in libraries, web-based delivery, preservation and metadata, and search and retrieval technologies.