Monthly Archives: August 2016

Using Make to Encode the Date and Time in BCD

I tried pretty hard several times over the past day to find a pre-built solution for doing this. This, being, program the “current” (to a few seconds) date and time into my STM32F4’s RTC for initialization, using a couple BCD words. Unable to find it (seriously? Does everyone program it from a user interface? I doubt that…), I had to create my own solution. My build tools being generally bash and make, I figured it should simply be a matter of setting date to the right output type. People make BCD clocks all the time, right?

date is a wonderful utility, but doesn’t have a BCD output (and probably shouldn’t since there are a million different ways to order the digits), so I just needed to process its output. Fine, this is fine. Giving date the argument +%-H,%-M,%-S tells it to output something like 20,12,43 to say the time is 20:12:43. My newfound friend awk can be given custom field delimiters through the -F flag (but for silly reasons some characters are better than others, I choose commas since they are generally safe), and generate a string as output. Unfortunately, the printf command in awk doesn’t have a way to print things as binary (hex and dec are fine). Sooooo, next utility at bat is bc, essentially a command line calculator with some nicer features than most unix command line builtins. Critically, it can convert numbers between arbitrary bases.

At this point, the general scheme is to have make do a shell call and grab the output. The shell call will be a call to date piped into awk which will build the command string to be piped to bc which will do the math and binary conversion necessary to get a set of decimal hours, minutes and seconds converted to a 32bit integer that matches the encoding for the STM32F446 (and potentially other chips in the STM32F4 line).

The final relevant make lines are as follows:

RTC_BCD_TIME := 0b$(shell date +%-H,%-M,%-S | awk -F"," '{print "obase=2;scale=0;hours="$$1";minutes="$$2";seconds="$$3";(((hours/10)*1048576)+((hours%10)*65536)+((minutes/10)*4096)+((minutes%10)*256)+((seconds/10)*16)+(seconds%10))"}' - | bc )
RTC_BCD_DATE := 0b$(shell date +%-y,%-m,%-d,%-w | awk -F"," '{print "obase=2;scale=0;years="$$1";months="$$2";days="$$3";dayofweek="$$4";if(dayofweek==0)dayofweek=7;(((years/10)*1048576)+((years%10)*65536)+((months/10)*4096)+((months%10)*256)+(dayofweek*8192)+((days/10)*16)+(days%10))"}' - | bc)

Note prepending the string returned by the shell call with “0b” to designate it as a binary sequence. I guess at the end of the day it didn’t need to be converted to binary, but it may help with debugging later on since it is BCD. It’s also worthwhile to be aware of the __DATE__ and __TIME__ automatically defined by gcc, but they are in string form and difficult to manipulate with just define statements. I felt like doing this outside the compiler was a better option. If anyone finds themselves in the same scenario, hope this helps!

Tee and Moira 2 (or, the Better Option)

So, last time I had found the moira interactive prompt utility which had a habit of just ultra-dumping my moira-list membership list along with a ton of control characters and excess prompt word schmoo. This is undesirable because any attempt to make a find-and-replace style moira list utility would require major cleanup of the logged output from the interactive prompt. However, since anything worth over-doing was once worth just plain doing, I knew there had to be a more naked utility hiding under the overall moira prompt.

Queue the MIT SIPB site, particularly its page on moira (*doh!*) Scroll to the bottom, you find “Making Moira Queries Directly,” and the acronym GLOM, get_lists_of_member. Perform a query (qy), of type glom (glom) to recursively find all memberships of a user (ruser) with the username (NAME) is the magic (mostly) answer!

qy glom ruser NAME

I say mostly because in this raw form it also releases a bunch of other information, but at least in an entirely repeatable (read, removable) fashion. However, more digging on the SIPB site shows that qy _help glom can give us more info on the glom-type query, its data-fields (importantly, list_name), and how to tell qy we only want the list_name field.

Appending -f list_name is the ticket, (again, mostly) to filter out the other data we don’t want. There is still “list_name:” prepended to everything, but at least a convenient colon is there to delimit.

Apparently, (typing as I’m working on this), -s is the final key! Add that and each list name is set nicely on its own line, without any fluff. Perfect!

For the next trick, a forum post tipped my off that awk is a useful utility for iterating through newline-delimited input. Typically, awk '/searchstring/ {print $0;}' inputfile is a silly way to grep-style search through inputfile and print any lines matching the regex searchstring. $0 is the “field” of the input line that matched searchstring for which you can craft the delimiters for. Luckily, I’ve already cleaned up the input so each line is a single string that is the only thing of interest, so everything here on out will use $0.

awk is also designed to use input files, and must be given “-” as the input file to be told to use stdin (e.g. to use piped input). so CMD | awk '/foo/ {print $0;}' - gets use well on our way.

Printing is very useful for debugging, but we want to actually do things at this point. The command sent to awk in the curly braces isn’t automatically a shell command; awk is an interpreter and has its own command set. Luckily it is easy to just pipe a print command through to shell: CMD | awk '/foo/ {print $0 | "/bin/sh"}' - will run whatever the matching lines are (which probably won’t be very useful).

Padding the print statement with some actual commands is the final step:
qy glom user NAME -f list_name -s | awk '{print "blanche "$0" -a LISTNAME" | "/bin/sh"}' -
This pipes the list of base moira lists of which NAME is a member, omits the optional filter in awk (processes all input), pipes the blanche command to a shell with the current line’s list as the first arg, then the -a option with LISTNAME as the moira member to add to the list.

Essentially, run this replacing NAME for your kerberos, and replacing LISTNAME for a moira list or member, to add LISTNAME to all lists you have permissions to modify and NAME is a member of. Obviously, you may not have permissions for every list you are a member of, but awk with print these errors and failover to the next line. Neat!

Tee and Moira

So, as all aging MIT students must, today I set about looking into adding my future email address to the mailing lists I’d like to remain on post-graduation. Obviously, it would be desirable to do so programatically rather than clicking through the Web Moira interface for the 700 lists I’m on an manually adding my new email to each. First things first, I know about and have used Blanche to add myself to lists and get info about lists. However, blanche doesn’t allow you to search by kerberos and get a list of memberships. The Googles allowed me to find a list of Athena utilities (including Blanche if you’re curious) that includes the Moira command-line interface!

It’s really a silly little interactive prompt that assumes you know way more about how things work than you probably do, but gets the job done in a somewhat unsatisfactory way. I’ll keep looking for options or a better utility. My work thus far:
1). Get yo’self an Athena dialup session or log in in a cluster *gasp!*
2). Pull up a terminal and enter moira
3). Option 3 for the Lists and Group Menu
4). Option 7 for the List Info Menu
5). Option 1 for the Show all lists to which a given member belongs
6). If you haven’t done anything crazy with how you logged in (you’ll know already if you are) the defaults should give you everything you ever wanted (just hit enter at each remaining prompt, it will autofill USER and your kerb name)
7). The output is a less-formatted spew of all the lists you are on!

This is great but I’m hardly going to copy paste this. It’s annoying that the output is embedded in an interactive prompt. I suppose a funny way to do it could be to run this inside screen, modify the apparent window height, then take advantage of the scrollback buffer and Copy Mode to scroll back to the start of output and copy all the lists to the screen buffer. But that’s gross.

So far, my target is still gross. I found out about tee which allows the interception of the usual stdin/out/err descriptors, with particular utility for logging them to files. Running:
moira | tee moira.txt
logs the output for the entire interactive session! Unfortunately, they used vt100 or similar terminal coding to blank the display, reset to home, and write the next prompt instead of whitespacing out the old stuff (for the record though, that’s 100% how you should do it, its just frustrating for the log files) so the logs are full of ESC[ gobbledeegook. oops. soooooo, automated cleanup, or a more better programmatic way of things? Next time!

Triangulating Camera Position from Known Points

triangulating camera position from known points opencv

Triangulating camera position from known points

As per my last post on this effort,, with the camera field of view parameters determined, and the lensing warp shown to be fairly low, there’s a straightforward path to taking an array of 2D points in the camera view, with known real-world coordinates, and backprojecting them to determine the camera’s position in space. Though this has probably been done a million times before by every vision system ever, it seemed like the kind of thing that should be easy! Or, as is often the case, would turn out to be interesting on its own and therefore worth the experience. 😀

Given my ultimate goal of extracting my swimming robot’s coordinates from a dual-camera setup, I need to know both cameras’ poses in the global reference frame to make a senseable coordinate extraction. So, with a static fishtank in the frame, containing the robot as well as generating perfect markers for a rectangular coordinate frame, I set about doing lots of trig. Essentially, the corners of the fishtank become a set of points in 3, with known coordinates since I can measure the fishtank. It’s a perfect rectangular prism, so it makes sense to align the global coordinate frame with its axes. Given 4 points on the fishtank, I can triangulate the camera location (and likely orientation though I haven’t thought that through yet and don’t need it).

Basic Method:
In OpenCV, click my 4 registration points on the 2D camera feed.
Given the pixel distance -> angle conversion I talked about in the previous post on the subject, convert every combination of 2 points to an angular measure.
Since the actual 3D position of all the points is known, the distance between any set of 2 is also known.
Taking each pair and considering the “point” location of the camera, it is clear there is a triangle for every point pair with the camera location as its third point.
The known distance between the two points on the fishtank is then opposite the angle determined by the pixel distance of the point pairs.
This should sound exactly like its heading towards the Law of Cosines to determine the other sides of a triangle with one angle known.
Since all of these imaginary triangles actually share sides with each other, a simple algebraic relationship exists to find all side lengths of all triangles from the known sides and the angles (these side lengths can also be interpreted as the distance from fishtank points to camera point).
To get back to what we actually want, the camera position in 3D, we can replace side lengths as values with side lengths as a function of the camera position. Instead of getting the lengths of every side, we do a little more algebra and get back from the solver the point that satisfies the distances.
If the world were perfect, we could stop there and call it a day. Just take the angles from the camera and the known line lengths from the fishtank points, run it through solver and boom, a point in ℝ3. If we were working with exact values, this could happen, but a lot of approximations have happened thus far (the actual dimensions of the fishtank, the selected points from the video feed, the assumption of a perfect pixel to angle conversion, among others).
So instead we are left with having to find the camera point that minimizes the error between the computer world and the real world. We can start from a test point in space, check how badly it fits our equations by comparing the triangles it makes with the edge lengths and angles we know, then try new points that make this error smaller. Its essentially a “hot or cold” search where you drag a point through space, getting constant feedback of “hotter” or “colder”.
Fortunately, this works great!

Some notes on this:
I first did a demo in MATLAB, since the visualization tools there are a little easier to use. There I discovered a few flaws, the biggest being that only optimizing for the point is globally stable, optimizing the distances of the camera to each point has some local minima that can trap the solver.
The second is that a non-linear optimizer like fmincon in MATLAB is really a math-package’s bread and butter, and they are not letting anyone peak under the hood. fmincon is /not/ available to the MATLAB Coder c-code generation utility. Bummer.
However, Free Software was to the rescue, with Python‘s SciPy package containing the desired non-linear optimization suite. minimize can take a scalar function of multiple variables and scrobble the inputs using various methods to find a minimum of the function. It’s really beautiful that tools this good are in the public domain.

Characterizing the PS3 Eye

Wikipedia claims that a PS3 Eye zoomed to “blue” has a field of view of 75 degrees. This is presumably the horizontal field of view while I need both angular measures, so I decided to check both out myself.

Put PS3 Eye sensor at 4.25″ above flat surface
Reinforce 8.5″x11″ sheet of paper with tongue depressors
Holding the paper vertically in Landscape, with one edge flush against, and exactly perpendicular to, the table, adjust angle of camera (it pivots about one axis on its base) and distance from paper until the top and bottom edges are at the very top and bottom of the camera view. The camera sensor should now be entirely parallel to the sheet of paper, in all axes. The edges of the paper should disappear and reappear together if you move the paper a little closer or a little further.
Mark the distance from paper to camera
Now shift the paper side to side, at this same distance, to observe where the side edges leave the frame. Mark the edges of the frame on the paper once the paper entirely fills the frame.

The field of view is then 2*atan((paper_measurement/2)/distance_to_camera), where paper_measurement is either 8.5″ or the distance between the horizontal extremes of the sheet that you marked.

This test resulted in a horizontal field of view of 60.32 degrees (not that I actually have that many sigdigs) and a vertical of 49.35 degrees.

It is important to note that points on a plane perpendicular to the camera can have their angles linearly interpolated from pixel distance against this maximum reference. The image resolution is 620×480 in normal video rate, so the horizontal angle between two points with deltaX=310 pixels, is 30.16 degrees. This should jive, as the image doesn’t appear overly warped. Horizontal distance is relatively the same anywhere on the sensor.

If the angle of the plane is not known, it is ambiguous whether a long object appearing short on the camera feed is doing so because of being at a sharp angle to the camera or because it is far away. The distance to one point must be known. However, perhaps a 3rd point, out of plane and thus defining a cube (like one of the back corners of my rectangularly prismatic fishtank) will provide the needed scaling factor. Should be cautious of accuracy here, if those point are at highly oblique angles.

Follow along with my next steps at locating the camera at

The MEAN Stack

Because this info is spread all over the web, and because I keep coming back to it after long hiatuses and having /no clue/ what I’m doing, here’s my own quickstart guide. Very much inspired after but sometimes more or less verbose

Install MongoDB:
sudo apt-get install mongodb may also work, but is at the mercy of repo managers to have the latest versions

Install Node: sudo apt-get install nodejs

Update npm globally: sudo npm install npm -g
Install Express-Generator globally: sudo npm install express-generator -g

setup app dir and install dependencies: express myapp
cd myapp
npm install

Make sure angular is setup in the app: npm install angular –save

at this point running: npm start
gives you your app at localhost:3000
where it should be available on the LAN for you viewing pleasure!

**********npm init*************
If you don’t want to use the automagical Express-Generator, you can use
npm init

entry point could be app.js

git repo!
optional keywords
license: GPL-3.0 is good
review and