μblog: engineering from the trenches

I have been reading James Gleick’s Chaos and I must confess that I am very impressed with the book so far. I am beginning to realize some of the practical applications of non-linear dynamics to analog circuit design, however, more on that later. What has been very interesting is the slow change in the mentality of the scientific world, from the notion that a small change in a systems initial conditions only warrants a small change in the output, to the reality that small changes in initial conditions can generate wildly different results. One of the pioneers in this field was the late Edward Lorenz. He discovered that a slight change (less than 1%) in the initial conditions of his deterministic weather model, which was numerically integrated, would cause the outcome to diverge from the unperturbed simulation to the point that the two weather systems were completely different after several days. The error in his integrator could not account for this disparity, therefore, he went through some analytic computations and found that simple differential equations can have very complex behaviors that were very dependent on initial conditions. He published his results in the Journal of Atmospheric Sciences, a paper that is well worth looking over. For those who are not mathematically inclined, looking over the introduction and conclusion should provide some insight into the paper. Additionally, this is the paper where the often duplicated Lorenz attractor, or butterfly attractor (figure 2) makes its first appearance.

( 1963lorenz-deterministic-nonperiodic-flow )

del.icio.us | digg

It took me a few months to finally read this book, but it was well worth it. I have been reading it prior to sleep as it was so full of information that it was difficult to read more than ten pages without taking a break to think about all of the new ideas. Furthermore, the information was presented in such an accessible manner that even those who are not specialists in relativity, topology or physics can appreciate the message.

I selected this book because I figured the topic was far away from electrical engineering that it could give a new perspective on understanding what is implied by measuring time and distance. Sure enough, this book provided many insights into the nature of our universe through the relation of time and space measurement. I will avoid summarizing the book, however, I will mention that it would be a pleasant read for those interested in non-Eucledian coordinates and the effects of gravitational fields. The book is extremely well written and reads much like a lecture series where the audience does not need to be able to carry out all of the steps of each operation, but acquires a taste for the process and a deeper appreceation. From the point of view of technical written English, this was one of the most understandable books on a physical subject that I have read in some time.

del.icio.us | digg

While looking for ways to escape muti-variate calculus purgatory in the final weeks of the semester, I came across Open Math Text.  These are a collection of math books (in PDF and LaTeX) that are openly available for distribution and are aimed at general scholars. A quick look at the collection will show that most of the books are authored by Dr. David Santos, a professor a the Community College of Philadelphia.  It seems that he has written and made available more books, in multiple languages, than the number of scholarly papers that most researchers publish at full universities.

While looking at his personal page, I found another open textbook collection called Textbook Revolution. The obvious downside is that these publications may not go through the same levels of review as textbooks printed at conventional publishers, however, it is nice to know that there is a group of people actively working to make affordable textbooks available.

del.icio.us | digg

As I have mentioned, the semester is winding down and projects are piling up. To help move things along, I have written a pair of documents that outline how to determine the conductivity of a plane (and volume) with a conducting disk (and sphere) of arbitrary size in the middle. I solved Laplace’s equations in both polar and spherical coordinate systems, then used boundary conditions to determine the electric potential and then determined the ratio of applied field to current density to determine the conductivity in the presence of the suspended object. I have checked these early drafts over a few times, however, there may still be some mistakes remaining, so please be warned. Also, feel free to post questions and I will make an attempt to answer them.

( disk-efield.pdf )

( sphere-efield.pdf )

P.S. The photo-op was staged during my last vacation, I would never use Classical Electrodynamics as a coaster.

del.icio.us | digg

The semester is winding down, which means there are a lot of deadlines piling up. One of my deadlines involves re-deriving some work by Hugo Fricke regarding the electrical properties of suspensions of conducting spheres in a conducting medium. Fricke was one of the pioneers of radio-therapy and was one of the first individuals to postulate that blood cells had membranes (instead of being homogeneous solids). He did this through electrical interrogation of blood alone without using any optical techniques. I am posting my step-by-step derivations for the electric potential inside and around a single conducting sphere in a conducting medium with regards to electrostatics. I solve Laplace’s equation using separation of variables in a spherical coordinate system. Hopefully I didn’t make many errors and the rest of the derivation relating total cell conductivity and capacity will follow.

( sphere-efield.pdf ) (Image is from GNU)

del.icio.us | digg

As promised before, I have finally worked through the majority of this paper, enough to give a brief introduction and discussion.

The key point of this paper is to demonstrate the importance of statistical analysis and its applications to determining information generation and transmission capacity. The measure H, or entropy, can be thought of as the amount of variance, or uncertainty, in a communication system. This leads us to define the theoretical capacity of a communication system given the known statistical properties of its constituents as well as apply analysis to practical systems.

The concept of information entropy deals with the uncertainty in the expected value of this information. Although it is rooted in statistical mechanics, it can be seen that highly predictable information has low variance, and therefore lower entropy, as compared to more random information. From this measure of information entropy, we can determine the necessary number of bits to efficiently encode this information, or to put it another way, how many symbols we can transmit per bit (assuming digital communication medium). Although the case of uniform probability distribution for all information symbols is easiest to analyze and leads to highest entropy, most practical applications have particular statistical distributions for symbol/information generation. Shannon goes to lengths to demonstrate this with the English language noting that selection of letters, or even words, is highly structured and far from random. This structure is a measure of redundancy of information, so that if I typ like ths, you cn stil undersnd me. (Spammers have been rediscovering this fact for years.)

Once the information entropy for all of the circuits involved in the communication system are determined, the channel capacity can be determined in the form of symbols per second given a finite certainty and a raw channel bit-rate. Shannon gives a fine example of a digital channel operating at 1000bits/s with a 1% error rate leading to an effective bit rate of ~919bits/s to account for error detection. Some communication system examples are given which I will not discuss in depth, however, I will try to reiterate the important steps in efficient communication design. Although Shannon gives a mathematical formulation for determining the theoretical limit for channel throughput, it is up to the designer to realize create a system which comes close to the limit. To do this, it is imperative to know the statistical properties of all of the sub-systems involved and the noise that may be present, and only then can efficiency be achieved.

The paper is by far more in-depth than this introduction and the math is not too hard, if anything, it is worth a look-over for some commentary on the statistical nature of the English language. As always, feel free to post a comment to discuss something about the paper, add something, or correct a mistake I have made. As a small bonus, I am adding Shannons’ patent for PCM-encoded voice/telephone service for those who like to read those types of things.

( 1948shannon-a-mathematical-theory-of-communication.pdf )
( 1946shannon-communication-syste-memploying-pulse-code-modulation-patent.pdf )

del.icio.us | digg

I have been working on spectral analysis of time series data and I am finding that (duh!) it is often easier to use someone else’s wheel than to invent a new one myself. That is, if their wheel has all of the necessary features and meets your performance criteria. I have been working with multi-taper spectral estimates and have found that the Chronux code (MATLAB scripts), available from Partha Mitra’s lab, to be fairly easy to use and effective for some continuous time-series analysis. The code also handles point process analysis, however, I don’t use that feature much.

The main reason that I like to use the multi-taper methods based on Slepian tapers, is that each taper gives a independent measurement of the signal spectra which then allows for computation of variances and therefore confidence intervals. This gives a certainty measurement to the analysis which can either make a strong case for its significance or will identify deficiencies in the data. Basically, these people put some years into writing this code and if you need to do coherence or spectral analysis in MATLAB, might as well give it a try.

del.icio.us | digg

Most scholarly articles that have some substance have a reference section at the end. The purpose of this section is to support scientific statements and to aid readers who would like to learn more about some of the article’s sub-topics. In rare occasions, authors use the reference section to show their scholastic diligence by citing some very old  or obscure sources. An example would be to include Laplace’s or Poisson’s equations in your paper, where relevant to electromagnetics, and then cite Maxwell’s Treatise on Electricity and Magnetism from the 1870s.  These are basic electromagnetics equations that are part of standard physics and engineering textbooks, which should be cited instead of Maxwell’s works. The rationale is that modern books are much easier to locate and are much easier to read. On the other hand, a good reason to cite Maxwell in a paper would be to discuss some particular part that is either unavailable or is contrasted to modern literature.

The purpose of this post is to present the best selling mathematical textbook in known history: Euclid’s Elements of Geometry. This contains a reproduction of the original Greek text along side a translation to modern English. Much of the contents of this tome are covered in typical high-school curricula, however, if you want to bolster your appearance of scholarship, this is probably one of the oldest works that anyone in the sciences will ever cite.

( elements.pdf )

del.icio.us | digg

Adding references, or data from, publications that are over a hundred years old seems to be a popular trend among scientific presentations these days. Sometimes it is to give a false sense of scholarship, however, it is often to remind us that some ‘new’ scientific breakthroughs may be simple re-interpretations of old discoveries. I try to note the references and look them up, when time permits. Here is the first paragraph from the preface of ‘Epilepsy and Its Treatments‘ (1904) by Spratling:

 The great progress made in the knowledge of epilepsy and its treatments during the past decade and a half, and in fact that no complete treatise on the subject has appeared in the United States since Echeverria’s work was published thirty-three years ago, was the chief reason that lead to the preparation of this volume.

With the exception of Manuel Echeverria (On Epilepsy: Anatomo-Pathological and Clinical Notes (1870)), the sentence can still be used in a modern book/review of Epilepsy without much alteration. The reason that the 1904 book was cited was to show that, a hundred years ago, physicians were aware that, on very rare occasions, were non-clinical. For example, it was noted that verbal interactions were sometimes enough to bring people out of seizure, something that researchers who seek alternative epilepsy treatments are rediscovering. (On a slight side note, there is an interesting personal account by Feydor Dostoyevsky starting at the bottom of page 466 where he links a pre-seizure state to a state of mental enlightenment.)

This long winded introduction was to present a pair of review articles from the early 1900s that covered what the authors thought were the highest achievements in physics and applied math of the previous century.

1905barus-the-progress-of-phsyics-in-the-nineteenth-century.pdf

1900woodward-the-centurys-progress-in-applied-mathematics.pdf

The image is from the IEEE and is of the Georgetown, CO steam/hydro powerplant.

del.icio.us | digg

This very small introduction to the Central Limit Theorem is probably something worthwhile before the Shannon paper. The main point is that as we take more and more samples from a random variable, with a fixed mean and variance, the samples approach a normal (Gaussian) distribution. That is, irregardless of the distribution of the random variable, if it meets the criteria, it will behave like a normal ly distributed random variable in the limiting case. The typical application engineering application of this theorem is making the assumption that some measured quantity is normally distributed and use that assumption to define things like confidence limits and so forth. The requirements for this assumption are that the process is second-order stationary, meaning the mean and variance do not change in the window of observation, and that the number of samples is approaching infinity. The requirement for a large number of samples can sometimes be loosened since the residual differences between the sample distribution and a normal distribution can sometimes be determined. The requirement for a stationary process cannot. For example, it would be foolish to apply Gaussian statistics to a random-walk (Brownian motion).

The key message is that the normal/Gaussian assumption is typically a good one, as long as the statistical nature of the random variable under investigation is constant through the period of observation and the number of samples is large.

( sec_4_f.pdf ) ( Image is from Wikipedia )

del.icio.us | digg