Stop The Madness

Figured I'd throw my 2 cents out on this: let's just get the hell out of Iraq, ASAP.

A Good Read

I just finished reading Bill Bryson's "The Life and Times of the Thunderbolt Kid". What a great book! It's basically about Bryson's childhood in the Mid-West in the 1950s.

Having spent my equivalent years growing up in New England in the 1970s, you might expect there to be a disconnect. Well, I'd just like to point out that there are some parts of New England, especially the more northerly bits, where I'm not sure the news of the revolution or continental congress have yet permeated. I'm not saying they're exactly backward up that way, and hey, even if I were, I'm entitled because remember, I said I'm from there, but, well, time just doesn't seem to flow the same rate everywhere, and someplaces, you get your time second or even third-hand.

So, there was much of "Thunderbolt Kid" that resonated. For instance, I especially like the bits about the screen doors, the "sleeping porch", and the pot-luck dinners. Also, I basically like pretty much everything Bryson writes, too, so that helps. On the whole, I'd recommend "Thunderbolt Kid" as a good read.

Neural Modelling

Here are some ramblings on "neural networks" I'd just like to set free ..

In the traditional neural network, neurons are typically modelled by computational nodes of various types. A node inevitably has some "output activation" level, and this is sometimes allowed to range from 0 to 1, in other models may range -1 to +1, and in so-called linear models can even take any value. At the same time, inputs to a node are weighted sometimes only with positive values, and in other models, positive or negative weights are permitted. While these seem overtly different, there is something they all have in common that is probably important.

What makes them all the same is this: you can take a snapshot in time, and at any single instant there will be some set of activation values for all the nodes in the network. Put another way, you don't have to observe over any finite length of time; an instantaneous observation provides knowledge about the activation of any and all nodes in the network. Since these observations are timeless (apart, maybe, from the fact of when the observation is made), I will refer to them as 'static' measurements.

This is not something you could do with real neural networks, i.e., real, biological neural tissue. The reason being that real neurons tend to fire in pulses. One could argue that the static measurements in artificial neural networks approximates real neurons by saying that "the (static!) activiation level of the artificial node at time T represents the average firing rate of the biological node in a narrow time window around time T." Well, maybe so, but a lot of information is lost this way. If you take the average firing rate over a sliding 100 millisecond window, you're really running the 'real' signal, the signal you're interested in investigating, through a low pass filter (LPF). Any effects based on the exact timing of individual pulses cannot be represented by such a "moving average". Furthermore, as the averaging window is narrowed, even though the LPF effect diminishes, the resolution, i.e., the number of distinct possible values diminishes, too. Widening the averaging window gives better resolution, but worsens the LPF effect. In the limit, as the averaging window is reduced to zero, the individual pulses reappear as items of consideration. It's not that the continuous-value, static activation approximation is not good for anything, it's just that it can't possibly capture ALL the possible behaviours that might be seen when individual firing pulses are considered.

For example, suppose you had a node with five inputs, with equal input weighting, and that each of these were firing at about the same LOW constant rate. Conceivably, you could have a node where, if all five of the inputs were to spike within a small interval, the node would fire its own output. Using the moving-average approximation, you'd be forced to say that "each input is at a fluctuating but SMALL value, and the node seems to fire somewhat randomly". If your averaging window was really small, you'd start to notice that the fluctuations would all be in the positive direction when the node fired and you'd have to conclude that the node was exquisitely sensitive to inputs. Well, with the averaging window made small, your 'fluctuations' would start to reveal individual input spikes, so you'd be onto something there, but in the static model, the necessary conclusion that the node is 'extremely sensitive' would force you to conclude that a single input, if amped up a bit, should suffice to tip the node into firing. Of course, amping up the 'level' of an input in the artificial model means increasing the frequency of firing in the biological model.

Well, I haven't worked out the details, but it seems that the static model isn't going to be able to produce all the behaviours of real neural systems, while real neural systems should be able to reproduce anything the static model is capable of. Sure, I need to put the math to it to make it a valid argument, but this is a BLOG isn't it? I'm just thinking out loud here. If someone does the math, or knows where it's already been done, and wants to tell me, "hey, yer full of it, look here's the math to prove it", that'd be welcome.

What If .. ?

What if cells started out without DNA? Sure, cells today are full of the stuff, but suppose that in the beginning, you just had these things that were just bags of metabolism? The basic dual-lipid-layer type cell membrane, enclosing a bunch of enzymes that run the Krebbs cycle or something? These early cells wouldn't have to do much, presumably they'd be floating in an environment rich enough in nutrients, all they'd have to do is just absorb them. Presumably also, the enzymes would be scarce enough, relative to the nutrients, that the entire environment wouldn't just metabolize itself out. All these DNA-less cells would have to do is absorb nutrients, expel waste products, grow, and when the surface area to volume ratio gets too low for efficient transfer, divide. Seems like something like that could form an ecosystem that could last a long time, if circumstances were right.

Then maybe, the odd bit of RNA or DNA could get trapped in some of these cells. It might not even do anything useful for the cell. Until the cell has an enzyme to duplicate it, you wouldn't even see the genetic material get amplified from division to division (though presumably, the enzymes to run the metabolism would already be getting reproduced, or daughter lines would just die when they ran out of these metabolic enzymes). Anyway, at some point, you'd find a cell containing both DNA (or RNA) and an enzyme that could duplicate it. Even if the DNA sequence produced no useful products for the cell, the cell would have to regulate, i.e., be able to switch on/off the reproduction of this DNA. Why? Assume there's no regulation. Either (A) the DNA isn't reproduced and some daughter lines cease to carry it, or (B) it is overproduced, and could kill the cell by virtue of just popping it .. sort of like a virus would do (hmm..), or (C) the cell could just get extremely lucky and produce just the right amount.

The reason I think metabolism preceeds genetics is just that, metabolic cycles are so much simpler compared to the processes involved in getting DNA duplicated, and especially when you consider that, unless some self-regulation is built in, getting it duplicated in the right amounts will be impossible, it just seems liklier that the simpler item came first. You can even see it as an evolutionary process, operating not at the level of DNA, but on the level of DNA: primeval DNA sequences that can't self regulate their own production either (a) fizzle out or (b) over produce and kill the cell, and so these don't tend to proliferate as time unfolds. Suddenly finding oneself in possesion of such unruly DNA is a sort of detrimental mutation, not within DNA, but at the level of "what the heck kind of chemistry should a cell include, anyway?".

On the contrary, a cell that finds itself host to DNA that can properly self-regulate at worst can be said to harbour a benign parasite, or you could call it a symbiosis, or you could view it as the birth of DNA based evolution, even if that genetic material isn't doing anything else useful for the cell, besides not killing it. Because, once well-behaved DNA exists like that, it's only a matter of time and useful mutation before that DNA starts doing something really useful for the cell, like producing more efficient enzymes for metabolizing different nutrients, or regulating the expression of enzymes to match changing conditions, and so on, in short, taking the bzillion or so steps necessary to become something that looks like a modern bacterium.

New Year's Resolutions

Ok, so, I'll put these things out into the blogosphere so they will have to come true, right?

2007 resolutions

1.Blogging
Edit less, post more. Editing seems to be the thing that mainly interferes with my posting here. I don't want to post unless I can find the time to polish it, but then there never seems to be the time to polish it, so I end up not posting it. Besides, to borrow from a famous cartoon philosophical moment, "you can't polish a ____, B". Yeah, I said edit less. I grant myself license to be cryptic. If you don't know who 'B' is, or what a '____' is, it really doesn't matter. Suffice it to say, there are some things you just can't polish much. So, forgave me sum speling and gramatical and errors there and there, and i promsie I will most pore.

2.Biking
I more or less neglected to mention, I think, that in 2006 I covered over 1400 miles of ground, mostly with a mountain bike, at average speeds of more that 12 mph. Yeah, you may scoff at a mere 12mph, but let me re-iterate, on a mountain bike. That was clearly nuts, so we (my wife and I) decided to get road bikes. So here's New Year's resolution #2: to ride at least three times that far on the new road bike in 2007. Preferably in 50 and 100 mile chunks, interspersed with loads of 30s. (For those biking-aware that may read this, I should mention that early results with the new road bike are very promising. Average speed has jumped from the 12-13 domain to easily 16+, and that's while being excessively careful, taking it easy trying not to tip over on the shiny new bike)

3.Oblique Angles
In 2007, I resolve not to make any ODD resolutions.

4.Recursion
In 2007 I resolve to make at least four resolutions.

5.Health.
It probably wouldn't be a Big List of Resolutions without something along the lines of "lose X number of pounds" or "quit detrimental vice Y", etc. Toward that end I will say only this: by the end of 2007 I expect to be more fit than I was when I graduated college. If you knew how I spent my senior year of undergrad, and/or read resolution #2 above, you'd see this, too, is probably a readily achievable goal.

6.Fly More Radio Control
In 2006, I learned how to take off and fly my R/C plane. In 2007, I resolve to learn how to land it, myself even. (In case anyone's worried, it's now hanging safely in one piece in my basement, waiting for warmer weather, thanks to the sure guidance of more senior members of the r/c flying club)