For “Machine Week” I was tasked with leading the software group, but none of these great people need any leading. I’m detailing my efforts below. Our goal is to write the software for an icosphere that can move forward by taking multiple steps in a given direction (orientation via IMU, 20 servos to actuate the sides of the icosphere). It should also play audio.
This was an incredibly busy week and it’s impossible to recall all the little tasks throughout the week. The outline below should provide a rough overview, though. More documentation can be found on the group page.
Tuesday
Sun and I met in the evening to figure out communication between a laptop as a remote control and an ESP32 on the icosphere itself. We opt for UDP as a communication protocol for speed. At the end of the night we have a setup that sends signals from the ESP32 via the local WIFI to a server running on a laptop. This server then passes data over to a website via websocket. The same works in the other direction.
Ended day at midnight.
Wednesday
The software team met in the evening to figure out some required components. I talked about 3D vector maths with Miranda, Saetbyeol and Yufeng. The problem to solve is how to maintain a consistent world space direction that we want the icosphere to move in while it rotates and changes its own coordinate system. It’s also the first time we get access to the IMU that the electronics team is planning on using. We’re getting familiar with the data coming out of the IMU.
End day at 10 PM.
Thursday
Participated in planning meeting at 9AM.
Morning: I spent time with Miranda to look at 3d vector maths theory. I had come up with a sequence of events that should get the maths right. We write it all up a bit more formally to prove to ourselves that it’ll do the trick. The plan: We’ll do a calibration once the IMU data settles, store the inverse of that quaternion and apply it to all future quaternion data coming from the IMU. That “zeroes” the icosphere’s coordinate system and all future transforms are relative to this zeroed state.
After dinner I catch up with the team, compile the latest ESP32 controller code on my laptop, and set up visualization scaffolding so I can work on that tomorrow.
End day at 2 AM.
Friday
Participated in planning meeting at 10 AM.
In the morning, I write all the visualization code and first version of the logic that figures out when to activate which triangles for a forward move. The logic is based on the visualization state (which is not ideal, but works for a project like this) and we use threejs vector libraries for our calculations. All of this works fairly well. Some problems had to be overcome like mis-matched coordinate systems (y-up vs z-up) between the IMU coordinates and threejs coordinates.
The icosphere data is brought into threejs via a glb that I exported from blender. The scene has triangle geometry representing the icosphere, as well as separate coordinate systems that are aligned with the triangles such that one of the axes is normal to the triangle surface. These coordinate systems are used in threejs to do comparisons with y-down and our movement vector to figure out which triangle the icosphere currently sits on and which it needs to roll over to. These comparisons are done via simple dot products between normalized vectors in world space.
End day at 11 PM or so.
Saturday
Participated in planning meeting at 10 AM.
Morning: Catch up with Miranda and explain all the code written yesterday. She can take over some of it and I don’t want to be a bottle neck. Otherwise the bus factor would be too high. She figures out some remaining math issues that have to do with axis alignment/flips.
We have many discussions around how to make the icosphere tip over an edge.
We then also have discussions around switching from UDP to BLE.
- It makes it much easier to connect to the ball.
- We observed latency issues when too many people were around (Sun did testing during StudCom tea where many people where present).
Afternoon: I provided some micropython and JS code (BLE) that I had written for one of the assignments to Miranda and she converted that over into our project (C++ and JS).
Evening: Work with Saetbyeol and Sun to make sure the servo code runs ok. We figured out the muxing. I ran down to the electronics lab to flip a bit on one of the breakout boards. That allows us to test all development IMU boards together because now they have different addresses on the I2C connection. The servos are working fine now.
Evening: Helped the hardware team with some lasercutting. Ceci has a minor accident and we take her to MIT Health.
End day at 4AM
Sunday
Planning meeting at 1 PM.
Discuss change of data stream from remote control to ESP32. Miranda implemented it. It is now a bit sequence (3 bytes total) with the first 4 bits depicting the control command sent over and additional bits to encode data. An example: A command to activate servos is sent via the control bits, the data bits signify which servos to activate. We make this change because we observed some BLE disconnection issues that we suspect have to do with the size of messages.
I helped the hardware team by cleaning one of the SLS prints. That printer produces that some great results! See below.
Miranda and I discussed the face -> [mux, channel] mapping with Ben and made a plan for how to approach this mapping once the icosphere is assembled.
End day at 2AM.
Monday
I make changes to our codebase to allow us to send 32 bits in the data stream driving the servos as opposed to just 20. We initially planned on sending over the triangle IDs that need actuation. Instead, we’ll now send over the channel on the muxes. It’ll help with debugging and shifts more of the intelligence over to the remote control. The idea is to have the ESP32 execute very simple commands. Since we don’t yet know the final mapping of triangle IDs to servo channels, we want to add that mapping in the browser code which allows us to iterate faster and without access to the physical microcontroller which might be built into the icosphere once we’re ready to figure out the mapping.
These changes make it easier to test the preliminary board that the hardware team provided as with, as well. With Dimitar, I tested their WIP board.
Tuesday
Miranda and I meet in the morning to discuss software changes. We’re trying to anticipate what we need for debugging once the hardware is fully assembled. We assume we won’t have much time to get it all running.
Miranda and I then helped Ceci shorten/grind metal rods. There was a slight problem with rods being too long leading to deformations within the icosphere that made it impossible to bring together all the parts as planned. Video below. Miranda cut the rods on the band saw, I remove the sharp parts on the belt sander.
We then get back to adding debugging tools to our software stack.
Everybody meets in the Mars Lab to help with the final assembly:
- I debug PCBs with Tyler. Our prepared software helps with figuring out servo channels on the PCBs.
- I cut servo arms, grind them, heat set them into the printed gears.
- I tighten many screws while helping assemble individual components that’ll make up the final system.
- I write some of the documentation
End day at 5:30 AM. The machine is built, it extends its triangles and it plays audio clips!
Short Reflection I don’t think I’ve ever been on a team with such amounts of talent. Doing this project with the group was such a joy, despite the sleep deprivation and the occasional stress. I’m very, very grateful to have had the opportunity to be part of this group and I’m so proud that not once have our high standards for kindness and support slipped. It was a stressful time, but everyone was incredibly committed and helpful. Such a cool experience!
