# blog.humaneguitarist.org

discoveries in digital audio, music notation, and information encoding

## trying to do a better job of image security

Just a quick post.

I've been thinking of image security lately, within the context on reading ebooks online.

Nothing online's going to be totally safe, but I have been thinking of better things I can do to protect an image on a website.

I've seen some sites that use image servers to protect access to the direct image, but if the image is called via the <img> tag, then all one has to do is use their browser's "Save web page complete" function.

I haven't investigated why, but calling an image via CSS' background-image property doesn't result in the image being downloaded via the browser.

I found a nice tutorial on "shrink wrapping" an image at http://skinnyartist.com/how-to-shrink-wrap-your-images/.

On top of that I used an image proxy script to read and return the data for a given referring URL only and used htaccess to block ALL HTTP requests to images within a given folder.

Here's the link to the "demo": http://blog.humaneguitarist.org/uploads/simijh/index.php. If you can download the image by hook or crook, I'd appreciate a comment below on how it was done. So far, using just Firefox, I can go to "View Page Info>Media>Save As" and get it although that's hopefully a bit of a pain and, therefore, a deterrent.

The PHP image proxy script and .htaccess file codes are below.

image.php

<?php

function return_image($image_url,$referring_url, $url_prefix="",$fallback_image="") {
/* Takes an image located at ($url_prefix +$image_url) and returns the image data provided the
HTTP_REFERER is equal to $referring_url. If the image does not exist it will fallback to the$fallback_image.

For the basic code related to proxying data in this way, see: "http://www.php.net/manual/en/function.fpassthru.php".
*/

// restrict access to image to $referring_url only. if ($_SERVER["HTTP_REFERER"] != $referring_url) { echo "You aren't allowed to see this image directly."; exit; }$image_url = $url_prefix .$image_url;
$binary = Null; // open the file only for .jpg. .gif, and .png files. if (stripos($image_url, ".jpg") == True
|| stripos($image_url, ".gif") == True || stripos($image_url, ".png") == True) {
$binary = @fopen($image_url, "rb");
}

// use the fallback image if opening the file failed.
if (!$binary) {$image_url = $fallback_image;$binary = fopen($image_url, "rb"); } // set the MIME type; send the image; stop the script.$extension = substr($image_url, -3); //will not work with extensions over 3 characters: i.e. "jpeg". header("Content-Type: image/$extension");
fpassthru($binary); exit; } // execute return_image(). if (isset($_GET["q"])) {
return_image($_GET["q"], "http://blog.humaneguitarist.org/uploads/simijh/index.php", "", ""); } ?> .htaccess <FilesMatch "\.(?:jpg|gif|png)$">
Order allow,deny
Deny from all
</FilesMatch>

February 8th, 2014 at 10:37 am

Posted in Uncategorized

## metadata is like the Muppets and Italian film

A couple of years ago, I was – against the better judgment of a friend – asked to give a guest class lecture at NC Central University on OAI, metadata harvesting, etc. for her metadata class at their school of library and information sciences.

Apparently, it went badly enough that I was asked the next year to try and get it right by that year's instructor.

I'm not sure if I did get it right after all, but I wanted to post the presentation slides all the same. They're here.

It's a little silly – using a Muppet motif throughout, but I think my main points got across. I think people are much more likely to get something if looked at in real-world terms (as if the Muppets are real). A little sense of humor doesn't hurt either.

I'd like to say that the "splash" page, a movie poster for Michelangelo Antonioni's film Blow Up, is the best part. It's all downhill from there, no doubt.

To me, the film made no "sense" until the last, surreal scene of people playing tennis without a ball put it all together for me. It was as if the entire film was a Subject and a string of Adverbs until the last scene, the Verb.

I let the class know this … and asked them to remember that while putting up with my presentation.

--------------

#### Related Content:

January 11th, 2014 at 11:20 am

## Monster Serials: listening to old radio plays while working on music notation

There are a lot of things I can do while having music, mostly classical, on in the background.

Working on music notation is not one of them. I don't mean composing the notes themselves – for that I need silence, but rather entering notes and performance indications into notation software.

So instead I like to listen to old time radio plays. Lately, I've had one of the following two playing in the background while I work:

Both are hosted on Archive.org.

Interestingly, the actor who portrays the monster in the Frankenstein piece mispronounces "Frankenstein" as "Franken-steen" initially, it seems. I had thought this was a subtle way of showing the monster getting smarter as he later learns to pronounce it correctly, but I believe some of the other actors made the same error here and there. So it might likely be just from the reality of reading a script and recording with limited time for checking for continuity errors.

I'll also mention that in trying to come up with a clever title (a play on Monster Cereals – get it?) I learned about Franken Berry Stool.

That is all.

--------------

#### Related Content:

January 4th, 2014 at 1:06 pm

## humaneguitarist.org enters the 21st century and some background information

I started my site (humaneguitarist.org) around 1996 after I thought it would be a good idea to share what I learned with regard to classical guitar posture.

After some major surgery in 1995, I tried to resume my studies with Christopher Berg at the University of South Carolina but I was simply too weak to do it. The telling moment came when I was in a practice room and dropped some sheet music. When I bent my knees to get down to pick the papers up, I was barely able to get back up.

On top of that, my back started to hurt like hell from practicing. I was using a footstool at the time because I found the A-Frame guitar support – widely used within our guitar program at the time – to be far too unstable for my tastes. The combination of a footstool and not being physically up to the task made for the perfect combination for a bad back. I'd wake up each morning feeling somewhat OK, but after a couple hours of practicing the pain became too great to continue.

Being too young to drink, I hit the books instead of the bottle.

And that's when the research that gave birth to this website began.

After some 17 years, I finally updated the HTML code … I don't even think CSS was around when I first created the site.

Now, of course it still looks the same but the code is much cleaner and easier to read. Mostly, it's not as annoying to me personally to know that it was messy under the hood.

Anyway, that's my story and I'm sticking to it.

Happy New Year.

Update, later in the day: Actually, thinking back the "humaneguitarist.org" domain probably goes back to only 1999. The main content has been online since about 1996 using America Online, GeoCities, tripod.com and whatever else until I got sick of changing URLs and finally started using my own domain name.

--------------

#### Related Content:

December 31st, 2013 at 12:40 pm

Posted in music,news

Tagged with , , , , ,

## discourteous accidentals

Earlier today I was proofreading the four pieces I'm writing to make sure I place courtesy accidentals where appropriate – as in where I deem them appropriate.

As the name implies, courtesy accidentals are just, well, a courtesy to the performer and technically aren't necessary.

In more chromatic sections, I think their importance rises. But I also think one has to try and combat instances where a musician might legitimately not know what to play – perhaps because the altered note is more ornamental in nature – or where that person might not know enough to make an informed decision as to what to play if there is a slight hint of ambiguiity.

In the excerpt below I was about to place a courtesy natural sign on the "g" in the bass voice of measure 3 as the "g" in measure 2 was a "g-sharp".

But then I realized I had already stated it was "g-natural" by specifying it would be played as an open string – considering that I didn't specify an alternate tuning. There are actually a few instances throughout the pieces where I think the fingering or implied fret position makes it clear what to play without the need for courtesy accidentals.

Of course, one could argue that I should place a courtesy accidental on the "f-natural" in measure 2 given that it was an "f-sharp" in measure one – albeit in a different voice.

But the guitarists in the audience already knew the correct note to play … right?

December 28th, 2013 at 1:23 pm

## LilyPond: explicit initial durations for a second voice and thanks to Frescobaldi

Happy Holidays, humbug, and stuff.

I'm in a coffee shop working on finalizing some pitches and durations for some pieces I'm working on for solo guitar. I'm hoping I can have the fingerings and dynamics all punched in by the end of the year – as I've got a nice amount of vacation built into the school year (one of the best advantages of working in academia). I started composing these last September (2012) so I'm behind schedule in that I wanted to be all done by September, 2013. Good intentions and all.

Anyway, I'm using the mighty Frescobaldi editor to make Lilypond scores after I'd done the initial notation in MuseScore and exported it to LilyPond format. And yesterday I was struggling with something I'd never come across before.

Basically I had thought that the first note defaults to a quarter note if no duration is specified. According to the LilyPond documentation here, that is indeed the case:

If the duration is omitted, it is set to the previously entered duration. The default for the first note is a quarter note.

The last measure of my first voice part ended with a half-note rest and the first note of my second voice part started with a quarter note, which I failed to explicitly notate because I thought a quarter note was the default per voice. My bad.

Well, it seems that the first note of the second voice was actually inheriting the half note from the end of the first voice so after getting a bit frustrated as to why my score was compiling all wonkily I stumbled upon the need to set the duration for the first note of the second voice explicitly as below.

  <g e' c'>4 r <b f' g> r |
<g c e>2\fermata r\bar "|." |
}

% voice 2.
basso = \relative c {
\voiceTwo
\stemDown
\set fingeringOrientations = #'(left)

c4 e r e | % 1
--------------

#### Related Content:

December 24th, 2013 at 10:55 am

## search and auto-complete suggestions with a little Solr and lots of SQLite

Over the last few days, I've been working on how to throw back search and auto-complete suggestions against a Solr index for an eBook project at work.

The idea is simple: a user starts typing and matches appear below the search box as the user types. The matches would "suggest" the best matching "title" and "author" values against the user query.

Conceptually, it's simple and quite easy to do with an SQL database. Problem is, we've – for now at least – decided to use Solr because it offers a far better fulltext search experience versus anything else we currently have experience with. We're in a bit of a time crunch, so exploring other options isn't much of, well, an option. For now.

Anyway, to do this I was just going to create a JSON API so our web developer could send the user's typed text to the API and get back the "best" matches. She's using typeahead.js in the mockup we have and it seems to be working well against a very small test index.

I should say that typeahead.js seems to support pre-loading a JSON file and doing a good job of showing suggestions against that file. The problem is that I don't see that scaling to potentially thousands of items, nor will that allow the most relevant matches to appear at the top. Pre-loading the data seems to simply follow the order of the items in the JSON file and it's impossible to know what the "best" order is for one user at any one time, let alone all users.

So, the API has to decide what to return and how to order it.

Anyway, I wasn't getting far with Solr's own "suggest" method – it really didn't seem to be what we wanted.

I also saw some tutorials that, honestly, I didn't have the time to explore and test with. The search-suggestion/auto-complete thing is, in the end, low priority. Very much unlike our deadline.

One tutorial here that was easy to work with and implement was based on using facets to return what I call those "perfect" matches. These are matches that are the (user string + a post-fixed wildcard). In other words, if the user types "the cat" a perfect match would be "the cat in the hat (title)", etc. At the end of the tutorial, note the warning about "the load caused by the use of faceting mechanism."

Anyway, the perfect matches alone wasn't really doing it for us; we also wanted relevant matches so that if one typed in "cat in the hat" the API would likely return "the cat in the hat (title)" because, while not a perfect match, it should have a high relevance score.

Well, in moments of crisis I like to remind myself of a little thing called SQLite and in-memory databases.

So, here what appears to be working. And it should scale, limited only by the speed of Solr but not directly by index size.

What I did was simply set the API to query against the "title" and "author" fields in Solr (with post-fixed wildcards).

I'm using the default of 10 rows for a search, so the maximum matches returned is 20, aka 10 * (# of fields). We can always bump that up if we think it's needed.

So after Solr returns the results, we have 20 total field values.

Those are then fed into an in-memory SQLite database.

So if the user typed in "the cat in th" the query would be a union of a post-fixed wildcard search and a fulltext search (with SQLite's built in support for Porter stemming!) against the 20 values. For the fulltext search each word in the user query would be separated by " OR ".

Since we need the results ranked by relevance, I hard-code the "perfect" matches to have a relevance rank of "99999" and then use the length of the "offsets()" function in SQLite to get the relevance for the remaining values.

After SQLite is done, the query results are simply placed into an array, with the field value added parenthetically – i.e. "the cat in the hat (title)". Then the array is JSON encoded and shipped out the door. By the way, I'm also making all the values lowercase for reading ease and to normalize things like "JANE DOE" and "Jane Doe".

p.s. just because 20 values are returned from Solr, that doesn't mean SQLite would return 20 results because many of those values are, of course, not perfect matches nor hits against the fulltext search. The remaining values could be returned as well, but so far I'm not thinking we want that as some of the results won't make sense to the user. That's to say if one types in "the cat in the hat" the value of "dr. suess (author)" could come back if we wanted it to as it would simply be the rest of the values obtained by adding another "UNION" clause in the database – i.e. a query that returns everything in the database. But the question is not "how?" but rather "why?".

Anyway, when/if a user clicks on a suggestion, the suggestion will be fed to a "search" method in the API that will sniff the field value out of the parenthesis and return results by using an " AND " separated search against that field in the index.

My only outstanding question at this point is whether I should do what I do now: insert each value into the SQLite database one at a time OR if I should build one big INSERT statement and insert all the data into SQLite in one fell swoop.

Anyway the code is below. Ultimately the meat of the logic here is Solr agnostic, since any data store with key/value pairs could be made to work with this.

Which is good because, as I initially stated, we're only using Solr out of time constraints not necessarily because we think it's the best way to go.

<?php

function return_suggestions($term,$min_len=2) {
/* Returns JSON auto-complete suggestions against Solr index of "author" and "title" fields for $term. Suggestions are lowercase and contain the field name within parenthesis. */ // force$term to be at least equal to $min_len. if (strlen($term) < $min_len) { exit; } // !!! these will eventually be handled by a commom library (query_tools.php).$term = trim($term);$term = str_replace("%20", " ", $term); // query Solr. include_once("includes/solr_tools.php");$params = "&q=author:$term*+OR+title:$term*&fl=author,title";
$response = query_solr($params, "select", "+OR+");

// get "docs" field from Solr.
$docs =$response["response"]["docs"];
//print_r($docs); //test line. if (!$docs) {
exit;
}

// create SQLite database.
$memory_db = null;$memory_db = new SQLite3(":memory:");

// create table; you must use "VIRTUAL TABLE" for fulltext (FTS3/4); see: http://www.sqlite.org/fts3.html#section_1_2.
$memory_db->exec("CREATE VIRTUAL TABLE box USING FTS4 (id INTEGER PRIMARY KEY AUTOINCREMENT, suggestion, suggestion_type, tokenize=porter)"); // get the "suggestion" and the field it came from; insert values into database.$fields = (array_keys($docs[0])); //the fields are "author" and "title" because those are the ones requested per$params/URL query.

foreach ($docs as$doc) {
foreach ($fields as$field) {
$suggestion_value = strtolower($doc[$field]);$insert = "INSERT INTO box (suggestion, suggestion_type) VALUES (\"$suggestion_value\", \"$field\")";
//echo $insert . "\n"; //test line.$stmt = $memory_db->exec($insert);
}
}

// prepare for query; make database query; run query.
$term_OR_separated = str_replace(" ", " OR ",$term); //for full text, need to separate words by "OR".

$query = "SELECT suggestion, suggestion_type, 99999 as rank FROM box WHERE suggestion LIKE '$term%'" //"perfect" matches.
. " UNION SELECT suggestion, suggestion_type, length(offsets(box)) as rank FROM box WHERE suggestion MATCH '$term_OR_separated'" //relevant matches. . " ORDER BY rank DESC"; //echo$query; //test line.

$results =$memory_db->query($query); // append matches to$suggestions.
$suggestions = array(); while ($row = $results->fetchArray()) { //print_r($row); //test line.
$suggestion =$row["suggestion"] . " (" . $row["suggestion_type"] . ")"; //i.e. "jane doe (author)" instead of "jane doe". if (!in_array($suggestion , $suggestions)) { array_push($suggestions, $suggestion); } }$memory_db->close();

// exit if $suggestions is empty. if (count($suggestions) < 1) {
exit;
}

// write and output JSON.
include_once("includes/make_json.php");
$output = array("suggestions"=>$suggestions);
echo make_json($output); } // execute return_suggestions(). if (isset($_GET["q"])) {
return_suggestions($_GET["q"]); } ?> Update, later in the day: So, I thought more about augmenting the query, but not to return all the remaining results but just ones for which the last word (or word fragment) entered by the user surrounded by wildcard characters yields something. Here's the part of the code I changed compared to that above (starting at line 47).  // prepare for query; make database query; run query.$term_OR_separated = str_replace(" ", " OR ", $term); //for full text, need to separate words by "OR".$term_last = explode(" ", $term); //geting last word in query; see: http://stackoverflow.com/a/11029470.$term_last = $term_last[count($term_last)-1];
//echo $term_last; //test line.$query = "SELECT suggestion, suggestion_type, 99999 as rank FROM box WHERE suggestion LIKE '$term%'" //"perfect" matches. . " UNION SELECT suggestion, suggestion_type, length(offsets(box)) as rank FROM box WHERE suggestion MATCH '$term_OR_separated'" //relevant matches.
. " UNION SELECT suggestion, suggestion_type, 0 as rank FROM box WHERE suggestion LIKE '%$term_last%'" //wildcard against last word only. . " ORDER BY rank DESC"; //echo$query; //test line.

$results =$memory_db->query(\$query);
--------------

#### Related Content:

December 7th, 2013 at 11:08 am

## too many brackets are being typed in the dark: XPath-like expressions for Python dictionaries and JSON

When I first started learning how to program and parse data, the data formats I first got acquainted with were text delimited files and XML. I avoided JSON as long as I could because I found (find?) it far less elegant than XML.

Actually whenever I read things where programmers put down XML because they think JSON has killed it, I'm not sure they have worked with some things where XML is probably a far, far better fit: like MusicXML.

I find XML infinitely easier to read . Not to mention XSDs and schemas a good thing even if they are a pain to write. But XPath is what I really like about XML. I hate it when people try to regex all over XML. I feel like saying, "Dude: this problem's already been solved!".

I prefer to write this for XML:

"foo/bar[1]"

to this for how I'd parse a JSON string after converted to a Python dictionary:

["foo"]["bar"][1]

Why?

Because I start typing really slow when intermxing brackets and quotations marks.
Maybe I should just work on my typing.

But for now, I'm playing with a little Python function called "jpath" that will allow me to use simple XPath-like expressions to return data from a Python dictionary a la:

>>> d = {"foo":{"bar":1}, "baz":[2,3]}
>>> jpath(d, "foo")
{'bar': 1}
>>> jpath(d, "foo", "type")
'dict'
>>> jpath(d, "foo", "text")
'bar'
>>> jpath(d, "foo/bar", "text")
'1'
>>> jpath(d, "baz", "type")
'list'
>>> len(jpath(d, "baz"))
2
>>> jpath(d, "baz", "text")
'[2, 3]'
>>> jpath(d, "baz[1]")
3
>>>

Below is a slightly more elaborate example where after pulling some data from the HathiTrust, the code parses the data using jpath() function.

from jpath import jpath
from urllib import urlopen

# make request for "Edgar Allen Poe" (sic) to Hathi Trust.
url = "http://chinkapin.pti.indiana.edu:9994/solr/meta/select/?q='edgar+allen+poe'&wt=json"
hathi = urlopen(url) #get response
hathi = loads(hathi) #convert to dict

### iterate through nodes within first item; print fields and values.
doc = jpath(hathi, "response/docs[0]") #get first item.
del doc["fullrecord"] #delete MARC data: too much XML to post in blog for a small example!
print ("*** Printing fields and values (as strings) for first item ...")
print
for each in doc:
print str(each) + ": " + jpath(doc, each, "text")
print

# print Title of first item.
print ("*** Printing Title of first item (as %s item) ...") %(jpath(hathi, "response/docs[0]/title", "type"))
print jpath(hathi, "response/docs[0]/title")

The script will output this:

>>>
*** Printing fields and values (as strings) for first item ...

mainauthor: [u'Poe, Edgar Allan, 1809-1849.']
htrc_gender: [u'male']
htrc_charCount: 346672
htrc_pageCount: 274
title_a: [u"Poe's poems"]
title_c: [u'[by] Edgar Allen [!] Poe.']
htrc_volumePageCountBin: M
htrc_volumeWordCountBin: M
oclc: [u'(OCoLC)16566920']
id: uc2.ark:/13960/t8pc2vk16
author: [u'Poe, Edgar Allan, 1809-1849.']
sdrnum: [u'sdr-ia-srlf2759906']
topicStr: [u'Poetics.']
title_top: [u"Poe's poems [by] Edgar Allen [!] Poe."]
publishDateRange: [u'1890']
htrc_wordCount: 58434
publisher: [u'The Henneberry Company']
author_top: [u'Poe, Edgar Allan, 1809-1849.', u'[by] Edgar Allen [!] Poe.']
publishDate: [u'1890']
countryOfPubStr: [u'United States']
htsource: [u'University of California']
language: [u'English']
htrc_genderMale: [u'Poe, Edgar Allan, 1809-1849']
title_ab: [u"Poe's poems"]
published: [u'Chicago, New York : The Henneberry Company, [189-?]']
title: [u"Poe's poems [by] Edgar Allen [!] Poe."]

*** Printing Title of first item (as list item) ...
[u"Poe's poems [by] Edgar Allen [!] Poe."]
>>>

And here's the "jpath.py" file containing the function.

# jpath.py

'''
to do:
>>> d = {"foo":{"bar":[1,2]}}
>>> jpath("//bar[1]")
2
'''

#####
def _jparse(_path):
''' Takes an XPAth like expression (nodes and positions only);
returns a snippet used to get data from a dictionary.

example:
>>> _jparse("foo[1]/bar[2]")
"['foo'][1]['bar'][2]"

'''

from re import split

# start final output string.
outpath = ""

# split by square brackets.
expression = split(("[$|$]"),_path) #see: http://stackoverflow.com/a/4998688

# work on writing "outpath".
for position in expression:

try: #leave integers in brackets alone.
test = int(position)
open_quote, close_quote = "[", "]"
except:
open_quote, close_quote = "['", "']"

# make string by splitting "_path" argument on forward slashes.
slash = "".join([(open_quote + slashes + close_quote) for slashes in position.split("/") if len(slashes) > 0])
outpath = outpath + slash #append to string.

return outpath

#####
def jpath(_dict, _path="", _format=""):
''' Takes a dictionary (name or string literal), an XPAth like expression (nodes and positions only),
and an optional output format ("type" or "text") and returns the value.

Returns an empty string if the expression fails.
'''

# parse "_path" to dictionary/brackets syntax.
outpath = _jparse(_path)

try:
# make final string to be evaluated; ex. >>> eval('{"foo":"bar"}["foo"]') #yields "bar".
dict_expression = str(_dict) + outpath
evaluated = eval(dict_expression)

# prepare output format per "_format" parameter.
_type = type(evaluated)

if _format == "type":
results = str(_type).split("'")[1]
elif _format == "text":
if _type == int:
pass
#evaluated = str(evaluated)
results = "".join(str(evaluated))
else:
results = evaluated

except:
results = ""

# return results.
return results

Yes, it's silly.
Yes, I should just work on my typing.

But I like it all the same.

Update, November, 11, 2013: I guess I should have taken a look at these first, BTW: https://pypi.python.org/pypi/jsonpath/ and JSONPath – XPath for JSON

--------------

#### Related Content:

November 10th, 2013 at 11:29 am

Posted in scripts,XML

Tagged with , , , ,

## Tomcat Murr and an apt depiction of the practicing “guitarist”

I'm currently reading E.T.A. Hoffman's The life and opinions of the Tomcat Murr : together with a fragmentary biography of Kapellmeister Johannes Kreisler on random sheets of waste paper.

There's a fantastic passage, not by Murr the Cat, but those rising from the random sheets of waste paper that, I think, are the best depiction of practicing I've ever read.

Thankfully, someone has already done the work of quoting the Penguin Classics English translation of the passage here, which I'll paste below. Notably, the quoter appears to be a pianist and their own introduction and conclusion to the passage are also worthy of note.

Aside from the quote is the equally interesting allusion a few paragraphs later to one Stefano Pacini, a luthier which the Penguin "Notes to Part 1" suggest "may have been the maker of Hoffman's own guitar."

Consider from page 41 of the Penguin edition the following passage …

The Princess overcame her alarm, and looked very closely at the instrument, whose strange shape would have shown its great age even had that not been confirmed by the date and the maker's name, which could clearly be seen on the bottom of the body through the hole in the soundbox, for the words 'Stepfano Pacini fec. Venet. 1532' were etched there in black.

Interestingly, while Hoffman lived in the 18th and 19th centuries, the "guitar" in the book is likely not what we now think of as a guitar. The "guitar" in this passage seems anachronistic to me as a 16th century guitar-like instrument was probably a vihuela or something similar in reality. In a fictional novel it can, of course, be whatever it needs to be.

Anyway, here's the original poster's quotation of the Hoffman passage …

My friends, I hope you will indulge me in a longish excerpt from E.T.A. Hoffmann's Lebens-Ansichten des Katers Murr from the Penguin Edition translated as 'The Life and Opinions of the Tomcat Murr'. It is one of the best descriptions of a musician's frustration I have read in a long time..

Quote:

Were the piano only light enough to lift, many are the times I would have cried thus and flung the thing out the window…

--------------

#### Related Content:

November 3rd, 2013 at 4:02 pm

Posted in literature,music

Tagged with , , ,

## PyEDS 0.3

Doh.

I just posted a PHP implementation of this and forgot I had updated the Python version.

This is small, light code, but it appears to be getting away from me in terms of versioning. Might be time to formalize it as a project.

#PyEDS.py

'''
This module provides a basic Python binding to Ebsco's EDS API, allowing one to:
- authenticate with a UserID and Password,
- open and close a session,
- perform a search,
- access the EDS Info and Retrieve methods,
- pretty print the JSON response.

Thanks,
Nitin Arora; nitaro74@gmail.com
____________________________________________________________________________________________________
#Usage example:

import PyEDS as eds

eds.openSession('PROFILE_GOES_HERE', 'GUEST_GOES_HERE', 'ORG_GOES_HERE')

#eds.authenticateFile() #alternative to using authenticateUser() and openSession()
#uses values in JSON config file argument(default="config.py")

#sample "config.py" file:
"""
{
"EDS_config": {
"UserId": "USERID_GOES_HERE",
"Profile": "PROFILE_GOES_HERE",
"Guest": "GUEST_GOES_HERE",
"Org": "ORG_GOES_HERE"
}
}
"""

cubs = eds.basicSearch('cubs')
piglets = eds.basicSearch('piglets', view='brief', offset=1, limit=10, order='relevance')

eds.closeSession()
____________________________________________________________________________________________________
CHANGELOG:
- version 0.3:
- added rawSearch() to support searching with full GET/URL string.
- this doesn't seem necessary but rather proper (the PHP version of this library requires it).
- version 0.2:
- downgraded version 1.1 to 0.1 ... I was getting ahead of myself!
- changed default file in authenticateFile() to "config.py" for security.
- changed data structures for everything except basicSearch() and advancedSearch() to be a
Python dictionary instead of a JSON string literal.
- fixed bug for "HighlightTerms" in retrieveRecord() as it needs to be an array, not a string.
- changed global list name to "__EDS" so as to use the Python convention for private names.
- made sure all functions end in a return statement.
- version 0.1:
- added retrieveRecord() and showInfo() methods.
- version 0.0:
- first version ... that worked! :-]

TO DO:
- add more options to basicSearch() like "facets", "search mode", "fulltext", "thesauras", etc.
- can't hurt! :-]
- consider adding an authenticateIP() function that uses the IP authentication method.
- deal with expired tokens, etc.; see: http://edswiki.ebscohost.com/API_Reference_Guide:_Appendix
- deal with any encoding issues re: Asian language characters returned from showInfo().
- add some error handling stuff.
- add examples for new functions up top.
'''

import json, urllib2
__EDS = {}

'''Authenticates user with an EDS UserId and Password.'''
auth_dict = {'UserId':UserId,
'InterfaceId':'WSapi'}
auth_json = json.dumps(auth_dict)
req = urllib2.Request(url='https://eds-api.ebscohost.com/authservice/rest/UIDAuth',
data=auth_json,
'Accept':'application/json'})
req_open = urllib2.urlopen(req)

req_results_dictionary = eval(req_results) #convert JSON to dictionary.
__EDS['AuthToken'] = req_results_dictionary['AuthToken']
__EDS['AuthTimeout'] = req_results_dictionary['AuthTimeout']
return

def openSession(Profile, Guest, Org):
'''Opens the EDS session with an EDS Profile, the Guest value ("y" or "n"), and the Org nickname.'''
sessionOpen_dict = {'Profile':Profile,
'Guest':Guest,
'Org':Org}
sessionOpen_json = json.dumps(sessionOpen_dict)
req = urllib2.Request(url='http://eds-api.ebscohost.com/edsapi/rest/CreateSession',
data=sessionOpen_json,
'Accept':'application/json',
'x-authenticationToken':__EDS['AuthToken']})
req_open = urllib2.urlopen(req)

req_results_dictionary = eval(req_results)
__EDS['SessionToken'] = req_results_dictionary['SessionToken'].replace('\\/', '/')
return

def closeSession():
'''Closes the EDS sesssion.'''
sessionClose_dict = {'SessionToken':__EDS['SessionToken']}
sessionClose_dict = json.dumps(sessionClose_dict)
req = urllib2.Request(url='http://eds-api.ebscohost.com//edsapi/rest/EndSession',
data=sessionClose_json,
'Accept':'application/json',
'x-authenticationToken':__EDS['AuthToken']})
urllib2.urlopen(req)
return

def authenticateFile(config_file='config.py'):
'''Uses values in config file to authenticate *and* open a session.'''
config = eval(config)
config = config['EDS_config']
openSession(config['Profile'], config['Guest'], config['Org'])
return

def rawSearch(query_string):
'''Returns search results using an ampersand-separated URL parameter string.'''
req = urllib2.Request(url='http://eds-api.ebscohost.com/edsapi/rest/Search?' + query_string,
'Accept':'application/json',
'x-authenticationToken':__EDS['AuthToken'],
'x-sessionToken':__EDS['SessionToken']})
req_open = urllib2.urlopen(req)
return req_results

def basicSearch(query, view='brief', offset=1, limit=10, order='relevance'):
'''Returns search results using basic arguments.'''
search_json = '''{"SearchCriteria":{"Queries":[{"Term":"%s"}],"SearchMode":"smart","IncludeFacets":"n","Sort":"%s"}, "RetrievalCriteria":{"View":"%s","ResultsPerPage":%d,"PageNumber":%d,"Highlight":"n"},"Actions":null}'''
%(query, order, view, limit, offset)

'''Returns search results using the full EDS search syntax (JSON).'''
req = urllib2.Request(url='http://eds-api.ebscohost.com/edsapi/rest/Search',
data=search_json,
'Accept':'application/json',
'x-authenticationToken':__EDS['AuthToken'],
'x-sessionToken':__EDS['SessionToken']})
req_open = urllib2.urlopen(req)
return req_results

def retrieveRecord(accession_number, db_code, terms_to_highlight=None, preferred_format='ebook-epub'):
'''Returns metadata (including abstract and full-text if applicable) for a single record.'''

retrieve_dict = {'EbookPreferredFormat':preferred_format,
'HighlightTerms':terms_to_highlight,
'An':accession_number,
'DbId':db_code}
retrieve_json = json.dumps(retrieve_dict)
req = urllib2.Request(url='http://eds-api.ebscohost.com/edsapi/rest/Retrieve',
data=retrieve_json,
'Accept':'application/json',
'x-authenticationToken':__EDS['AuthToken'],
'x-sessionToken':__EDS['SessionToken']})
req_open = urllib2.urlopen(req)
return req_results

def showInfo():
'''Returns EDS limiters, sort options, search tags, expanders, and search modes available to the profile.'''
req = urllib2.Request(url='http://eds-api.ebscohost.com/edsapi/rest/Info',
'x-authenticationToken':__EDS['AuthToken'],
'x-sessionToken':__EDS['SessionToken']})
req_open = urllib2.urlopen(req)
return req_results

def prettyPrint(json_string):
'''Returns a pretty-printed, UTF-8 encoded JSON string with escaped non-ASCII characters.'''
return json.dumps(dictionary, ensure_ascii=True, indent=2, encoding='utf-8')

#fin
--------------

#### Related Content:

October 13th, 2013 at 5:31 pm

Posted in scripts

Tagged with ,

Switch to our mobile site