While the controller itself is incredibly basic, the bulk of the work will be integrating the arm into ROS (which I want for future extensibility). Setting up a driver node and the connection from that ROS node to the driver “firmware” running on the Staubli controller will be an interesting task.

The Staubli CS9 controller uses it’s own language to give integrators access to hardware registers, control modes, set destinations, and interact with the GUI on the user “pendant” for guided applications. The language, called VAL3, is vaguely C-like but without any indirection, and logical blocks sort of like Basic. It’s a little nerfed, I think to avoid programming errors that are a little more dangerous in this context than just on a computer. But very functional at the small overhead of more functions and variables than “optimal”.

There is an existing ROS integration for Staubli robotic arms, but it is very basic, exposing only point-to-point motion capabilities and relying on Staubli’s controller for all path-planning and lower-level performance. This is a nice abstraction for pick-and-place applications, but defeats my eventual goals for force control at the end-effector. Trajectories often take upwards of 200ms to clear even for sub-millimeter motions, since the controller has to plan, execute and verify the trajectory.

Instead, I’m going to take advantage of one of the advanced features in the controller (free for evaluation if you’re ok restarting the controller every once in a while), a function called alter. Originally designed for sanding/buffing/polishing applications, it allows applying an offset to the current trajectory. This allows “injecting” feedback into the controller from other sources, where otherwise it would operate rigidly on its own encoders and that’s it. The key twist is that the trajectory for the controller is not deemed complete until alter mode is exited. In this way, we can start a trajectory with a tiny epsilon and immediately enter alter mode, and from there control our position directly from ROS until we wish to exit. We can perform our own analysis of termination criteria, run our trajectory control live, and incorporate whatever sensing we want into the final control effort output. Perfect!

date is a wonderful utility, but doesn’t have a BCD output (and probably shouldn’t since there are a million different ways to order the digits), so I just needed to process its output. Fine, this is fine. Giving date the argument +%-H,%-M,%-S tells it to output something like 20,12,43 to say the time is 20:12:43. My friend awk can be given custom field delimiters through the -F flag (but for silly reasons some characters are better than others, I choose commas since they are generally safe), and generate a string as output. Unfortunately, the printf command in awk doesn’t have a way to print things as binary (hex and dec are fine). Sooooo, next utility at bat is bc, essentially a command line calculator with some nicer features than most unix command line builtins. Critically, it can convert numbers between arbitrary bases.

At this point, the general scheme is to have make do a shell call and grab the output. The shell call will be a call to date piped into awk which will build the command string to be piped to bc which will do the math and binary conversion necessary to get a set of decimal hours, minutes and seconds converted to a 32bit integer that matches the encoding for the STM32F446 (and potentially other chips in the STM32F4 line).

The final relevant make lines are as follows:

RTC_BCD_TIME := 0b$(shell date +%-H,%-M,%-S | awk -F"," '{print "obase=2;scale=0;hours="$$1";minutes="$$2";seconds="$$3";(((hours/10)*1048576)+((hours%10)*65536)+((minutes/10)*4096)+((minutes%10)*256)+((seconds/10)*16)+(seconds%10))"}' - | bc ) RTC_BCD_DATE := 0b$(shell date +%-y,%-m,%-d,%-w | awk -F"," '{print "obase=2;scale=0;years="$$1";months="$$2";days="$$3";dayofweek="$$4";if(dayofweek==0)dayofweek=7;(((years/10)*1048576)+((years%10)*65536)+((months/10)*4096)+((months%10)*256)+(dayofweek*8192)+((days/10)*16)+(days%10))"}' - | bc)

Note prepending the string returned by the shell call with “0b” to designate it as a binary sequence. I guess at the end of the day it didn’t need to be converted to binary, but it may help with debugging later on since it is BCD. It’s also worthwhile to be aware of the DATE and TIME automatically defined by gcc, but they are in string form and difficult to manipulate with just define statements. I felt like doing this outside the compiler was a better option. If anyone finds themselves in the same scenario, hope this helps!

Queue the MIT SIPB site, particularly its page on moira (doh!) Scroll to the bottom, you find “Making Moira Queries Directly,” and the acronym GLOM, get_lists_of_member. Perform a query (qy), of type glom (glom) to recursively find all memberships of a user (ruser) with the username (NAME) is the magic (mostly) answer!

qy glom ruser NAME

I say mostly because in this raw form it also releases a bunch of other information, but at least in an entirely repeatable (read, removable) fashion. However, more digging on the SIPB site shows that qy _help glom can give us more info on the glom-type query, its data-fields (importantly, list_name), and how to tell qy we only want the list_name field.

Appending -f list_name is the ticket, (again, mostly) to filter out the other data we don’t want. There is still “list_name:” prepended to everything, but at least a convenient colon is there to delimit.

Apparently, (typing as I’m working on this), -s is the final key! Add that and each list name is set nicely on its own line, without any fluff. Perfect!

For the next trick, a forum post tipped my off that awk is a useful utility for iterating through newline-delimited input. Typically, awk '/searchstring/ {print $0;}' inputfile is a silly way to grep-style search through inputfile and print any lines matching the regex searchstring. $0 is the “field” of the input line that matched searchstring for which you can craft the delimiters for. Luckily, I’ve already cleaned up the input so each line is a single string that is the only thing of interest, so everything here on out will use $0.

awk is also designed to use input files, and must be given “-” as the input file to be told to use stdin (e.g. to use piped input). so CMD | awk '/foo/ {print $0;}' - gets use well on our way.

Printing is very useful for debugging, but we want to actually do things at this point. The command sent to awk in the curly braces isn’t automatically a shell command; awk is an interpreter and has its own command set. Luckily it is easy to just pipe a print command through to shell: CMD | awk '/foo/ {print $0 | "/bin/sh"}' - will run whatever the matching lines are (which probably won’t be very useful).

Padding the print statement with some actual commands is the final step:

qy glom user NAME -f list_name -s | awk '{print "blanche "$0" -a LISTNAME" | "/bin/sh"}' -

This pipes the list of base moira lists of which NAME is a member, omits the optional filter in awk (processes all input), pipes the blanche command to a shell with the current line’s list as the first arg, then the -a option with LISTNAME as the moira member to add to the list.

Essentially, run this replacing NAME for your kerberos, and replacing LISTNAME for a moira list or member, to add LISTNAME to all lists you have permissions to modify and NAME is a member of. Obviously, you may not have permissions for every list you are a member of, but awk will print these errors and failover to the next line. Neat!

It’s really a silly little interactive prompt that assumes you know way more about how things work than you probably do, but gets the job done in a somewhat unsatisfactory way. I’ll keep looking for options or a better utility. My work thus far:

1). Get yo’self an Athena dialup session or log in in a cluster gasp! 2). Pull up a terminal and enter moira 3). Option 3 for the Lists and Group Menu 4). Option 7 for the List Info Menu 5). Option 1 for the Show all lists to which a given member belongs 6). If you haven’t done anything crazy with how you logged in (you’ll know already if you are) the defaults should give you everything you ever wanted (just hit enter at each remaining prompt, it will autofill USER and your kerb name) 7). The output is a less-formatted spew of all the lists you are on!

This is great but I’m hardly going to copy paste this. It’s annoying that the output is embedded in an interactive prompt. I suppose a funny way to do it could be to run this inside screen, modify the apparent window height, then take advantage of the scrollback buffer and Copy Mode to scroll back to the start of output and copy all the lists to the screen buffer. But that’s gross.

So far, my target is still gross. Enter tee which allows the interception of the usual stdin/out/err descriptors, with particular utility for logging them to files. Running:

moira | tee moira.txt

logs the output for the entire interactive session! Unfortunately, they used vt100 or similar terminal coding to blank the display, reset to home, and write the next prompt instead of whitespacing out the old stuff (for the record though, that’s 100% how you should do it, its just frustrating for the log files) so the logs are full of ESC[ gobbledeegook. oops. soooooo, automated cleanup, or a more better programmatic way of things? Next time!

As per my last post on this effort, characterizing the ps3 eye, with the camera field of view parameters determined, and the lensing warp shown to be fairly low, there’s a straightforward path to taking an array of 2D points in the camera view, with known real-world coordinates, and backprojecting them to determine the camera’s position in space. Though this has probably been done a million times before by every vision system ever, it seemed like the kind of thing that should be easy! Or, as is often the case, would turn out to be interesting on its own and therefore worth the experience. :D

Given my ultimate goal of extracting my swimming robot’s coordinates from a dual-camera setup, I need to know both cameras’ poses in the global reference frame to make a sensible coordinate extraction. So, with a static fishtank in the frame, containing the robot as well as generating perfect markers for a rectangular coordinate frame, I set about doing lots of trig. Essentially, the corners of the fishtank become a set of points in ℝ³, with known coordinates since I can measure the fishtank. It’s a perfect rectangular prism, so it makes sense to align the global coordinate frame with its axes. Given 4 points on the fishtank, I can triangulate the camera location (and likely orientation though I haven’t thought that through yet and don’t need it).

Basic Method:

In OpenCV, click my 4 registration points on the 2D camera feed.

Given the pixel distance -> angle conversion I talked about in the previous post on the subject, convert every combination of 2 points to an angular measure.

Since the actual 3D position of all the points is known, the distance between any set of 2 is also known.

Taking each pair and considering the “point” location of the camera, it is clear there is a triangle for every point pair with the camera location as its third point.

The known distance between the two points on the fishtank is then opposite the angle determined by the pixel distance of the point pairs.

This should sound exactly like its heading towards the Law of Cosines to determine the other sides of a triangle with one angle known.

Since all of these imaginary triangles actually share sides with each other, a simple algebraic relationship exists to find all side lengths of all triangles from the known sides and the angles (these side lengths can also be interpreted as the distance from fishtank points to camera point).

To get back to what we actually want, the camera position in 3D, we can replace side lengths as values with side lengths as a function of the camera position. Instead of getting the lengths of every side, we do a little more algebra and get back from the solver the point that satisfies the distances.

If the world were perfect, we could stop there and call it a day. Just take the angles from the camera and the known line lengths from the fishtank points, run it through solver and boom, a point in ℝ³. If we were working with exact values, this could happen, but a lot of approximations have happened thus far (the actual dimensions of the fishtank, the selected points from the video feed, the assumption of a perfect pixel to angle conversion, among others).

So instead we are left with having to find the camera point that minimizes the error between the computer world and the real world. We can start from a test point in space, check how badly it fits our equations by comparing the triangles it makes with the edge lengths and angles we know, then try new points that make this error smaller. Its essentially a “hot or cold” search where you drag a point through space, getting constant feedback of “hotter” or “colder”.

Fortunately, this works great!

Some notes on this:

I first did a demo in MATLAB, since the visualization tools there are a little easier to use. There I discovered a few flaws, the biggest being that only optimizing for the point is globally stable, optimizing the distances of the camera to each point has some local minima that can trap the solver.

The second is that a non-linear optimizer like fmincon in MATLAB is really a math-package’s bread and butter, and they are not letting anyone peek under the hood. fmincon is /not/ available to the MATLAB Coder c-code generation utility. Bummer.

However, Free Software was to the rescue, with Python’s SciPy package containing the desired non-linear optimization suite. minimize can take a scalar function of multiple variables and scrobble the inputs using various methods to find a minimum of the function. It’s really beautiful that tools this good are in the public domain.

Put PS3 Eye sensor at 4.25” above flat surface, when the camera tilt is relatively neutral

Reinforce 8.5”x11” sheet of paper with tongue depressors

Holding the paper vertically in Landscape, with one edge flush against, and exactly perpendicular to, the table, adjust angle of camera (it pivots about one axis on its base) and distance from paper until the top and bottom edges are at the very top and bottom of the camera view. The camera sensor should now be entirely parallel to the sheet of paper, in all axes, since the midpoints are lined up. The edges of the paper should disappear and reappear together if you move the paper a little closer or a little further.

Mark the distance from paper to camera

Now shift the paper side to side, at this same distance, to observe where the side edges leave the frame. Mark the edges of the frame on the paper once the paper entirely fills the frame.

The field of view is then 2*atan((paper_measurement/2)/distance_to_camera), where paper_measurement is either 8.5” or the distance between the horizontal extremes of the sheet that you marked.

This test resulted in a horizontal field of view of 60.32 degrees (not that I actually have that many sigdigs) and a vertical of 49.35 degrees.

It is important to note that points on a plane perpendicular to the camera can have their angles linearly interpolated from pixel distance against this maximum reference. The image resolution is 620x480 in normal video rate, so the horizontal angle between two points with deltaX=310 pixels, is 30.16 degrees. This should jive, as the image doesn’t appear overly warped. Horizontal distance is relatively the same anywhere on the sensor.

If the angle of the plane is not known, it is ambiguous whether a long object appearing short on the camera feed is doing so because of being at a sharp angle to the camera or because it is far away. The distance to one point must be known. However, perhaps a 3rd point, out of plane and thus defining a cube (like one of the back corners of my rectangularly prismatic fishtank) will provide the needed scaling factor. Should be cautious of accuracy here, if those point are at highly oblique angles.

Follow along with my next steps at locating the camera in space.

Final Presentation: Pink: Origin from 2.009 @ MIT on Vimeo.

Final Presentation: Pink: Origin Q&A from 2.009 @ MIT on Vimeo.

I couldn’t be more proud of everyone on Pink Team, it was an incredible semester and a ton of fun. GO PINK!

I’ll probably post some more technical stuff soon, but I have to get back to psets.

Otherwise, another spoiler is appropriate:

But hang on, we haven’t talked at all about the product design process! And what an exciting and challenging process it is.

The ideation process taught in 2.009 is Professor Wallace’s preferred method, and is the same as the one taught in 2.00b. Essentially, the first round of the process is to gather a many ideas as possible, without filter, to amass a huge board of thought-provoking and not-provoking ideas. Participants should be encouraged to generate derivatives and spin-offs, along with fresh material. From there, a brief grouping process can show connections between ideas, and optionally filter ones that were off topic. With some clearer categories, a more focused brainstorming session can generate new concepts, flesh out old ones, or remix a couple. At this point, attention should be paid to the core concept of an idea; what problem is it trying to solve? The first implementation that comes out on paper may not be the best.

From here, we took a fairly democratic path by voting on concepts that seemed interesting and tractable to test, and finally developed a Pugh Chart to try and select three concepts impartially for initial testing. The cooler here was one of my concepts that reached the initial stages of testing (and we continued pursuing it until the final vote). Some other popular ideas we worked on were actively regulated thermalwear, an ice depth testing device, and a portable light-based sports field line marking system. After several stages of testing, mixing in new ideas and performing more tests with old ones, Pink team reached a point where we had three strong, tested, and exciting ideas. A pivotal moment for every team in 2.009 is the meeting where the final product is decided. Luckily, Pink team was full of intelligent, impartial, critical thinkers, and the decision making process went smoothly (and relatively quietly!).

What I said about the first implementation not being the best? It’s probably true quite frequently, and it was certainly true for our final product, Origin. Originally (hah! get it?), another team member and I came up with fairly similar concepts for a wrist-mounted proximity beacon to alert sky divers of others around them, in hopes of preventing canopy collisions (the #1 cause of death in skydiving). Initially, people were concerned about the market size and utility of such a device. At some point, someone considered the broader need; location information for social/safety/datametrics reasons in areas outside the range or use cases of cell phones. That was the moment of birth of the idea that grew into Origin, our final product.

Hopefully now you are wondering, “What is that?”, “Where can I buy one?”, “How did you guys get to test the next [tech company] wearable?”. Tuck yourself in, and I’ll tell you the story of 2.009

One of MIT’s best known courses, 2.009 takes over 150 MechE seniors through the beginning stages of product design, up to the point of presenting a (hopefully!) functioning prototype product to a live and streamed audience of a couple thousand. So highly esteemed is the course on campus, within 2 hours of the RSVP system going live, every available seat in Kresge Auditorium (1,200 according to the campus activities site) was claimed. There’s one reason for the hype, and I can tell you that us students deserve no credit in the class culminating in this incredible final event.

The course is taught by Prof. David Wallace, who pours his heart and soul into making the class an unrelenting experience. As soon as the class is sectioned into 8 teams of 24 (yes, there were 24 people in a team), we start doing team bonding tasks that are cleverly disguised as learning activities. One of our first tasks as Pink Team was dismantling a product and determining the materials used, how it was assembled, and trying to determine the materials cost of the product. These activities were way more enjoyable than your typical name-game group activities, and really began to shape the team dynamic. We learned early on who was loud, who to trust, who had good ideas, etc.

Later we worked on assembling a strange contraption manufactured by the shop staff. An earlier lecture, we had been asked to sketch out an idea for a multi-person vehicle that could load and carry cargo without the operators touching the cargo or the ground. I had sketched out what was essentially two bikes, side by side, with a bucket in the middle, and a scoop to pile cargo in. The shop staff apparent found this idea entertaining, so they built it, with some modifications as you can see. Our team assembled it, then a couple of us took it for a test drive. We knew this wasn’t the last we would see of it.

It certainly wasn’t. The next week, each team took their mobile, stuff-getting, people-transporting device out to Killian Court, the home of the iconic Dome. Today, however, it was home to tiny volcanoes, stacks of blocks, and hidden treasure! The teams were given a crate of pom poms and some other paraphernalia with which to cheer on their team as pairs went out to uncover their treasure and bring home the gold. We had been told ahead of time that there would be a prize for first, but also a prize for spirit. Knowing that our dual-bike thingy “hobbled” at best, as a team we decided to strike out for the team spirit prize. We started preparing for the event hours before! I pulled together some cruft foam insulation to make giant Pink letters, found some surplus scrap wood from Next Big Thing, and took along some rope. Another division went to a nearby fabric store to get a couple yards of pink cloth. We ended up building a 20-foot, guy-line supported Pink flag, which we proudly displayed during the competition. It was a huge team bonding moment because, although we came in dead last getting the treasure, we had a ton of fun being the loudest and proudest out there. It was a good moment, and I knew every member of the team was in it for the right reasons.

Enough about team bonding, and on to the technical stuff!