Gary Robinson's Rants

Calculating the product of a Python list

I came across a stackoverflow question about multiplying the numbers in a Python list. Suggestions there included:

def product(list):
    p = 1
    for i in list:
        p *= i
    return p

and

from operator import mul
reduce(mul, list)

You can also do:

import math
math.exp(sum(map(math.log, list)))

It's not as readable as the reduce-based solution, though if you're a mathematician who isn't familiar with reduce() the opposite might be true!

I wouldn't advise using the log-based solution under normal circumstances. But if you're ever in a situation where you risk overflow or overflow, such as in

>>> reduce(mul, [10.]*309)
inf

and your purpose is to compare the products of different sequences rather than to know what the products are, then

>>> sum(map(math.log, [10.]*309))
711.49879373515785

is the way to go because it's virtually impossible to have a real-world problem in which you would overflow or underflow with this approach. (The larger the result of that calculation is, the larger the product would be if you could calculate it.)

Selenium's WebDriver & Python: Dealing with script timeouts

Selenium's WebDriver is a great way to interact with web sites in an automated way. It's primarily for testing, though I've also had occasion to use it for other purposes (with the permission of the relevant site owners).

In order to interact with sites that use a lot of Javascript, it's helpful to use Selenium's Firefox driver. It works great, but I did have one problem, where scripts were timing out. You can modify Firefox's timeouts via changing preferences, but Selenium's Firefox driver uses its own profile, so it ignores Firefox's usual means for changing preferences.

I searched the web for solutions and found various suggestions that didn't work. Perhaps many of them worked with pre-WebDriver versions of Selenium, I don't know.

In any case, the following worked for me to change the timeouts to 10 minutes:

from selenium import webdriver

profile = webdriver.firefox \

.firefox_profile.FirefoxProfile()

set_pref = profile.set_preference

set_pref('dom.max_script_run_time', 600)

set_pref('dom.max_chrome_script_run_time', 600)

driver = webdriver.Firefox(fx_profile=profile)

Note that the dom.max_chrome_script_run_time preference has nothing to do with Google Chrome, it's related to Firefox's internal Chrome URL's. In fact, for my purposes, I only needed to change dom.max_chrome_script_run_time.

Note: if you're not familiar with WebDriver, do help(webdriver.Firefox) for a description of the methods you have available for interacting with sites. Hint: the oddest thing is that to fill in a text field, you get an object representing the element with the text field, and then call myElement.send_keys("the text").

Update: Docs on the Python bindings may be found here.

 

Profiles In Courage

 

I've been paying particular attention to one Egyptian who's been tweeting about events, whose Twitter ID is Sandmonkey. I don't know about you, but when I start following somebody via blogs or twitter and they are expressing what matters to them, I feel a bit like I know them. I start to care about them. Here are his most recent tweets (skipping the ones I can't read because they're in Arabic, and the ones that only link to another page):

Sandmonkey 
Anti-Mubarak protesters are descending on downton in the thousands. They are not giving up Tahrir. #JAN25
16 hours ago Favorite Retweet Reply

Sandmonkey 
I hear reports of Army evacuating the Square from Protesters. Is this true? #jan25
5 hours ago Favorite Retweet Reply

Sandmonkey 
Either way, I am heading there with medical supplies. They better not block my entrance. #jan25
5 hours ago Favorite Retweet Reply

Then somebody telling him how to get into Tahrir:

ahmada2 
@Sandmonkey it's open. Kasr el Nile or talaat harb.
5 hours ago Favorite Retweet Reply

Then…

SorayaBahgat 
Reports about my dear friend @Sandmonkey being arrested in#Tahrir. So worried can anyone give me more info? @bencnn#Egypt
1 hour ago Favorite Retweet Reply

and a number of other tweets from different people reporting the same thing, including:

lisang Lisa Goldman

.@sandmonkey was arrested today & his blog has been suspended. Read his very important final post about #jan25 here: http://bit.ly/hqIJ2b

That post ends:

"This is a losing battle and they have all the weapons, but we will continue fighting until we can’t. I am heading to Tahrir right now with supplies for the hundreds injured, knowing that today the attacks will intensify, because they can’t allow us to stay there come Friday, which is supposed to be the game changer. We are bringing everybody out, and we will refuse to be anything else than peaceful. If you are in Egypt, I am calling on all of you to head down to Tahrir today and Friday. It is imperative to show them that the battle for the soul of Egypt isn’t over and done with. I am calling you to bring your friends, to bring medical supplies, to go and see what Mubarak’s gurantees look like in real life. Egypt needs you. Be Heroes."

Update: He is out of jail now. "He was roughed up, but is fine." Sitting here in Bangor, Maine, enjoying the peaceful, beautiful snow, I'm in awe of this kind of courage.

Later update:

Sandmonkey Sandmonkey

I am ok. I got out. I was ambushed & beaten by the police, my phone confiscated , my car ripped apar& supplies taken #jan25

28 minutes ago Favorite Retweet Reply

Sandmonkey Sandmonkey

will tell the story later . Thank you all. I just need to rest now. #jan25

16 minutes ago Favorite Retweet Reply

 

Last update, Feb 11:

The courage of Sandmonkey and all the protestors has paid off. Unbelievable to have been privileged to watch this history unfold in real time. Here's the last tweet from Sandmonkey I'll post here:

 

Sandmonkey Sandmonkey

 

To everyone who rediculed us, opposed us, wanted us to compromise, i say: YOU ARE WELCOME :) TODAY WE ALL CELEBRATE!!! #JAN25

 

Now, let's hope that the ruling Supreme Council of the Armed Forces doesn't turn out to just enable a continuation of the same type of authoritarian regime...

 

 

 

 

 

 

How to tune a guitar

...if you're a luthier, that is.

I was wondering why a Yamaha classical guitar I purchased a few years ago always seems out of tune, and started a discussion about it in a guitar forum, including measuring the pitch while each string is open and fingered on the 5th and 12th frets (bizarrely inconsistent results). They suggested I check out this fascinating article by Gregory Byers. It describes the lengths (pun not intended but allowed) a guitar-maker has to go to to create a situation where each string is somewhat in tune on each fret. My Yamaha was not made that carefully! Of course very few guitars are. Byers spends a month making a guitar,  and they cost $9,500.

Get yer mental health here!

A friend of mine, Tom Sulcer, with whom I like to harmonize on tunes like the Beatle's "If I Fell," posted a fairly massive article called "Mentally Healthy Mind" in a Google knowl. It touches on everything from deliberative democracy to Maslow to Spinoza. Did I mention that it's massive? My own impression is that most of the important understandings in life are largely unconscious and hard to modify through conscious deliberation, whereas Tom appears to try and leverage a conscious understanding of just about everything related to living a life. We'll see who's right by noting which of us plays better guitar at age 95.

A Dog's Consciousness

Here's a way to imagine what it would be like to be a dog.

Take a few moments to imagine yourself exactly as you are now, with the exception of not being able to focus your attention on anything that exists only in your head. You don't even have to imagine not having thoughts. You just can't focus on them.

Non-blocking raw_input for Python

[Edited Aug. 30, 2010 to fix a typo in the function name and generally improve formatting]

I needed a way to allow a raw_input() call to time out. In case it's useful to anyone, I wrote this solution which works under Unix-like OS's.

import signal

class AlarmException(Exception):
    pass

def alarmHandler(signum, frame):
    raise AlarmException

def nonBlockingRawInput(prompt='', timeout=20):
    signal.signal(signal.SIGALRM, alarmHandler)
    signal.alarm(timeout)
    try:
        text = raw_input(prompt)
        signal.alarm(0)
        return text
    except AlarmException:
        print '\nPrompt timeout. Continuing...'
    signal.signal(signal.SIGALRM, signal.SIG_IGN)
    return ''

 

 

Snow Leopard Guest User data loss bug

I've seen a number of mentions today of a bug that can cause a Snow Leopard user to lose all their data:

The problem appears to manifest itself on machines which had the Guest account option enabled under Leopard and were subsequently upgraded to Snow Leopard. Users booting their machines have reported that upon start-up, they have been logged into the Guest account. Upon switching to their regular account, the affected users have been finding all of their user data missing and unrecoverable except from a backup. [MacRumors.]

One user reports a way to recover the lost data. Since it's buried in a discussion thread, I'm reproducing it here (also fixing a typo pointed out by a reader of this blog):

The files were still in /Users,
recovery was not too difficult.

1) su in terminal
2) mv username username.old
3) create account username
4) mv username username.new
5) mv username.old username
6) chown -R username username

I don't know whether it's a general fix. If you run into the bug and decide to try this solution, feel free to post your experience in the comments section!

FlyFi.com upgrade

We've made a substantial upgrade to the look and feel of our Flyfi.com web site. The site does a lot, and this release has been geared toward making the functionality more readily apparent and easier to use.

FlyFi Community Playlist

If you're a Facebook user, you might want to check out my company's FlyFi Community Playlist app which lets a Facebook community collaborate on a playlist... but it's more than a playlist because it also provides tracks related to the manually entered ones.

A fair number of the tracks are legally downladable free MP3's, from superb artists like Randy Newman (one of my personal top 5 artists), Ani DiFranco, Suzanne Vega, Dar Williams, and many others.

Correction re singletons

The source code link presented at the main post for my singleton mixin classes was bad until now. It pointed to an earlier version that didn't have thread safety or various other improvements. Sorry.

Even more singleton improvements

singletonmixin now deals with cases where you want to be able to include args in calls to S.getInstance() even after the initial instantiation.

More singleton improvements

Based on internal work and feedback appearing in comments on this blog, I've updated the Python singleton class I posted way back in 2004. That earlier post remains the place to get the code and place any further comments.

It now handles keyword arguments, and is threadsafe for singleton creation.

Cable News

I was stunned at how poor the Iran coverage was on cable news over the weekend. Then I saw this thought from Matthew Yglesias:

Whenever I find myself talking about new media to skeptics of an older generation who worry that the standards online are too debased, I try to remind people that the real debasing came with the rise of multi-channel cable news. In terms of the Iranian elections, the world’s top newspapers have the people on the ground reporting the main facts, and there’s lots of smart analysis from legitimate experts all over the web, but on television if it can’t be captured by two talking heads debating each other it’s like it never happened. (Hat tip to Jason Linkins, who also provided the emphasis.)

I had noticed that CNN seems to show an awful lot of discussion between experts over each event -- often much more of that than direct coverage of the event itself But I'd never thought of it in quite the way Matthew puts it. Anyway, I turned on CNN this morning to see whether, for a change, they were covering Iran.

And they were -- kind of. There was some talking head being asked about whether Obama should be taking more of a stand in favor of the protestors. The guy's opinion was that he should.

The picture is being coming clearer. Generally CNN consists of one or more talking heads in a room, taking turns talking about whether the latest news event is being handled in accordance with their personal opinion. Proportionately speaking, there seems to be not all that much actual reporting of facts.

I still don't get the reason why this is happening. Maybe it's because people find it more enjoyable than actual reporting because it's a little like gossiping about the failures of the newsmakers? Or is it that and there's a tiny bit of "human drama" when the talking heads disagree -- kind of a combination of news and reality TV? Or could it simply be that it costs less than actual reporting? (They probably don't even have to pay the talking heads, who are happy to be there to publicize themselves.)

Iran

If you've been following developments in Iran, and particularly the Internet's essential role in reporting (and supporting) events, I don't have much to add.

If you haven't, I strongly suggest you go to Andrew Sullivan's blog and starting following him now -- as well taking a bit of time to look through his posts from the last couple of days. He's been doing a wonderful job. Far better than any mainstream news outlet, with the possible exception of the NY Times' Lede. (Although the Lede doesn't present as much detail and analysis as Sullivan does -- with the help of his readers and well-chosen links to other sources.)

One of Sullivan's readers sums up the key role Twitter is coming to play:

Ahmadinejad's and Khamenei's websites were taken down yesterday - I saw the latter go down within a couple of minutes because of a DDOS attack organised via Twitter. @StopAhmadi is a good source for tweets on this. The other important use of Twitter has been distribution of proxy addresses via Twitter. This would be how most video and pictures of today's rally have gotten out.

One amazing thing is that it seems at least conceivable that the protests would not have been as successful as they currently appear to be if Twitter didn't exist.

Sullivan suggests wearing green in support of the protesters. I think that's a great idea.

Flyfi.com

Most people reading this blog will know that I am a founder of the company that makes Flyfi.com (basically a vastly improved reboot of the old Goombah site).

In the last couple of weeks we have added enormously to the site, including improved recommendations. If you haven't checked it out, please do!

Here's a note I wrote to introduce it to some high school friends who I recently reconnected with on Facebook:

Hi Everyone. You may remember me playing my guitar in the hallowed halls of B.H.S. In college I became interested in math & computer technology, and in recent years, I've come full-circle by finally merging my interests. I've co-founded a music-oriented Internet startup. Our product is FlyFi.

Our VP/Industry Relations who, in a former life, earned 10 Grammy nominations as a producer, is now dedicated to using his industry connections for getting us the highest quality free music collection on the Internet. We've got great indie artists like Ani DiFranco, some well-known, others up-and-coming. Unlike some of our competing services you don't just hear the music for free -- you can also download many tracks as free legal MP3's.

My area of focus is our music recommendation technology, which lets you type in artists you like and get music out that you'll probably also like. I love music from many genres, from the Beatles to Beethoven to John Coltrane to Leonard Cohen. But I think there are underlying commonalities in the music I most love, regardless of genre. So we have statistical algorithms that try to find music that has those commonalities for each person's tastes.

Over the last 6 weeks or so, I've been completely wrapped up in putting the finishing touches on the first full release of FlyFi (which is why some of you may have noticed that my Facebook communication flow has dropped pretty much to zero). It's out now, ready for your perusal.

So, if you love music, go check out FlyFi.com!

Best,
Gary

Tom Swifties

One of my more unusual pastimes is making up Tom Swifties. Examples of my efforts in this area include:

"She said I look like a common farm animal!" Tom said sheepishly.

and

"Those damned cannibals! They're seasoning us with lemon juice!" Tom said sourly.

The NY Times is having Tom Swifty contest this weekend. Submissions include:

"My men will never mutiny," said the Captain blithely."

In the esthetic of the Tom Swifty, I believe that significant demerits apply to those where the description of the act of speaking doesn't really match the tone of what is said, such as this Times submission:

"She's got my photo in her locket," said Tom independently.

and this one:

"What's under this green jello," Tom asked sublimely.

Here are two particularly fine submissions, from Chris Doyle:

"I'll just have to kill the king," Reggie sighed.

"May I have this dance?" Fred asked gingerly.”

Inspiration from Miles, Herbie, and John

From an interview of Herbie Hancock and John Mclaughlin; they'd been discussing Miles Davis:

HH: We're not going to get rid of problems. I think there's a great beauty to having problems. That's one of the ways we learn. This is something I learned from Miles, first. I had this experience that I've talked about a lot of times. I was playing with Miles and we were doing this concert in Stuttgart. This was one of the nights when the band was hot. The stuff was burning, Tony Williams was smoking, Wayne wasä JM: Scrambling those eggs. HH: And sweating. And Miles was just playing like God-like stuff that he played. It was just smoking. And then, at this one point, which was like a peak in Miles' solo, I hit this chord that was so wrong. It was just awful. It was in the wrong place and it was like boom, I just felt like I destroyed the music. And Miles took his breath and played some notes that made my chord right. JM: Damn. HH: I don't know where he found these notes but he just wiped away the chord being wrong. He made this chord fit. I was dumbfounded. I couldn't even play for about two minutes. He just blew me away and what it taught me was that Miles didn't hear it as a mistake. He just heard it as an event. He just trusted it and did his musician thing and found the notes that fit that thing. I said, wait a minute, this is a lesson not just for music but for life. Things that happen to you are events. It's what you do with them that determine whether they're going to be problems or solutions. This is the kind of thing that I hope to develop more in my life and spread. And it's not something for just musicians, it's something that everyone can spread. JM: I'm on your side. We've got to encourage responsibility. We're all in it together. We need each other desperately. Now more than ever.

Einstein and this blog

Er, no actual relationship there. But

a) This is one of the most wonderful short articles I have ever read. It's about one journalist's brief but enormously rich encounter with Einstein.

b) My hypothesis in my previous post has turned out to be correct; when I type my name into Google, I am the top hit again.

Google vs. this blog

This blog used to be very highly ranked on Google. But as of today, I can't find it there at all. Instead, it finds my old blog, which I haven't updated in years. But that one did get a huge number of hits at one point, due to posting some anti-spam mathematical ideas. So that may be the reason for its still having a decent Google rank.

I have been extremely busy with FlyFi, and posting on this blog has simply become a lower priority. One problem is that much of what I spend my time thinking about is business and technology ideas for FlyFi, which are not yet appropriate to share publicly!

Anyway, I was wondering whether, simply by posting this on my blog today, Google will see more activity and therefore make me findable again.