Flip It!

INTRODUCTION

The Flip It is a fun hand-eye coordination game that tests one’s ability to accurately flip a phone-sized rectangular prism to match a randomly generated series. It utilizes the FRDM board’s on-board accelerometer/magnetometer (the FXOS8700CQ 6-axis sensor [8]) to detect when it is in freefall, and then uses magnetometer data from the same device on the board to detect flips along its major and minor axes.

Flip It

[Fig. 1: This image shows the Flip It device]

Acc and Mag

[Fig. 2: This image shows the onboard FXOS8700CQ 6-axis sensor]

This project was selected due to its lack of need for external hardware or complicated interface with a PC. We did not want to have to rely on shipments of any kind nor the processing power of our computers, so the project requires only an external battery pack which is a common every-day item that many already own. We also both wanted to be able to test the project without needing to buy two sets of hardware tools, since we are working remotely.

Though the template provided for I2C communication was heavily leaned on, we learned a lot about this type of communication scheme while modifying the provided code to output magnetometer data. We also learned a lot from the tutorial on UART communication, and its speed limitations (especially at 9600 baud), while using the USB line to debug our program, and analyze our flip-detection technique.

The biggest lessons came from analyzing magnetometer and accelerometer data. Since the accelerometer data cannot be relied upon in free-fall, the magnetometer data was used as a three-directional compass to analyze flip conditions. This is because the accelerometer is a device which detects apparent forces on an object, not the net acceleration vector directly. An object in free-fall has no such support, and therefore no apparent weight [5]. Choosing the conditions that must be met to register a flip was difficult, but in the end we were able to register different types of flips reliably, while rejecting flips that were not sufficiently “pure” as described further below.

Our game logic was designed to make a simple, random test of someone’s memory and ability to perform flips in series. Through designing this, we learned about the difficulty of simultaneously gathering data for accurate operation of the device while trying to interface with humans, which are relatively slow creatures requiring long LED blinks (compared to other processes that must occur) to understand what is expected of them.

Due to the intermediate axis theorem [6, 7], flipping our device around its minor axis and major axis was possible, while rotations around the third axis are unstable. Therefore, only flips along the y-axis (a.k.a “pitch”-flips or “hot-dog”-flips) and flips along the z-axis (a.k.a “yaw”-flips or “flat”-flips) are requested from a player. The project was a great success and the game, while difficult to play, is fun nonetheless. While our original idea involved more complicated scheduling requirements, simplifying the schedule made the game more consistent and user-friendly. We were able to not only design a program which can accurately detect different kinds of flips and a program which can detect free-fall, but also a fun game that any owner of a FRDM K64f board (and a battery pack) can enjoy!

XYZ axes

[Fig. 3: This image shows the Flip It with the appropriately labeled axes]

SYSTEM OVERVIEW

Block Diagram

[Fig. 4: This is a block diagram of Flip It. The accelerometer and magnetometer are both part of the FXOS8700CQ 6-axis sensor, but they are shown as separate to make their roles clearer. Note that the communication protocol used is I2C.]

[Video Link 1: Flip It! (Note: This is our main video.)]

SYSTEM DESCRIPTION

Data Collection, Free-fall Detection, and Flip Detection:

Interfacing with the Magnetometer and Accelerometer:

To read data from the FXOS8700CQ 6-axis sensor [8], we first watched the course staff tutorial on the Inter-Integrated Circuit (I2C) communication protocol and began working from the posted code-base provided [1]. In order to view that data was read correctly, we also used the Universal Asynchronous Receiver/Transmitter (UART) functionality, provided in the same code base, to print the data to our PC’s using PuTTY Serial console. At first, we simply observed the raw data, performed a number of flips, and realized that the accelerometer data was only useful for letting us know when the board was in free-fall.

Following the FXOS8700CQ datasheet example for driver code [9], we set up the device for performing 200Hz hybrid data collection from the accelerometer and the magnetometer. This meant adding I2C_WriteReg() function calls to 3140_accel.c to set the two magnetometer control registers using the same slave address (same device), as well as modifying the accelerometer control register settings to work in hybrid mode. Another function was added below ACCEL_getAccelDat called ACCEL_getMagnetDat, which again followed the datasheet instructions to store magnetometer readings from the correct positions in the data buffer coming from the device and store them in an SRAWDATA struct. Since both accelerometer data and magnetometer had x, y, and z data registers, we simply made two separate structs to hold the data in separate bins. While clearer for us to understand with two separate requests to the I2C device for each type of data, speed could be increased by combining these functions to a single request to fill six fields in a struct.

Free-fall Detection:

By taking the magnitude of the acceleration vector while the device is on a table or is otherwise static, and subtracting that value from the magnitude of subsequent accelerometer readings, we acquire a signal which is 0 when the board is held still, spikes when it is thrown, is constant and non-zero while in the air, and spikes again when caught before returning to zero again. This is consistent with the physics of motion, since the board must be sped-up to be released, and slowed-down to be caught, producing acceleration spikes.

By setting a free-fall flag when the magnitude of this signal is sufficiently high, we can consistently detect if the board is in the air, provided that the offset magnitude of gravity while the board is sitting still was taken accurately. By setting another flag on the free-fall flag’s falling-edge, we can determine when the board has been thrown and then caught, which constitutes a single attempt at a flip.

Flip Detection:

In order to observe the orientation of the board, we first explored calculations used in the aerospace industry for acquiring the pitch, roll, and yaw angle of the device [2, 3]. Using magnetometer data, we attempted to convert these raw data-readings into an angle of the board in its own coordinate system. We expected a signal which spikes in only one data channel (pitch, roll, or yaw) while performing a flip in that one direction. Unfortunately, the signals we observed from these calculations were not as useful as we thought. Because division is required to calculate angles using trigonometric functions, artifacts of these calculations appeared in a number of example data traces and attempted flips. Also, because more complicated and careful calculations are required to take the raw data, convert them to angle calculations, and then map them to an appropriate range, this method was abandoned.

Because of our dependence on the magnetometer, our game needs to be oriented in a certain direction with respect to the Earth’s magnetic field in order to operate correctly. However, this does not affect the quality of gameplay.

To smooth otherwise noisy data collection, a history of 10 readings is maintained for the acceleration vector magnitude, and the x, y, and z magnetometer readings. The average of the 10 most-recent readings is the value used in detection, and is also what is displayed in figures 5 and 6. The derivative of each set of data is approximated as the most-recent reading minus the 10th most-recent reading.

Observing the raw magnetometer data is sufficient to detect flips of the types that we require. During a “pitch”-flip, where the y-axis is roughly pointed in a constant direction, the z-axis reading should “spike” above a threshold amount, and return to zero. The x-axis, on the other hand, should go through a “zero-crossing” of sufficient magnitude, as the Earth’s magnetic field vector sweeps through it. A zero-crossing is determined by a separate function which returns a 1 if 1) the history of the length-10 input array shows a change in sign between the last and the first entry and 2) the derivative is above a user-input threshold amount. This ensures that while small sign changes in the input data will be safely ignored, the rather quick zero-crossing of a real in-game flip will register as such [4].

Similarly, during a “yaw”-flip, where the z-axis is roughly constant, the y-axis “spikes” above a certain threshold, while the x-axis also goes through a zero crossing of sufficient magnitude.

During a flip along the intermediate axis, or “roll”-flip, the x-axis readings are roughly constant, while the z-axis “spikes” and the y-axis goes through a zero-crossing. This y-axis zero-crossing is used to ignore bad flips with this behavior.

Data Collection:

Magnetometer Readings

[Fig. 5: Magnetometer readings throughout flips about each axis.]

Derivative Calculations

[Fig. 6: Derivative calculations throughout flips about each axis.]

Thresholds which determine these conditions were experimentally derived, but good starting values used data directly from the board output using the provided UART connection. The data was copied to a text file from PuTTY, imported into Excel, and then plotted to generate graphs shown. Using this algorithm, only sufficiently “pure” flips in a single axis direction will be detected. Flip It requires a lot of practice for a high score!

GAME LOGIC

Button Setup:

We needed some user interaction to allow different stages of the game logic to cycle. Since we don't want to use any other hardware, we used the in-built button SW3 [10] for triggering the orientation cycle and distinct rounds of the game. This button is mapped to PTA4, so the System Clock Gating Control Register 5 (SCGC5) register 9 must be enabled. PTA4 is then set as a GPIO input pin, and its IRQC fields (in the Port Control Register) are cleared and set to 1001, which triggers an interrupt when a rising edge is detected at this port. The ISR is then enabled and implemented at the top of gamify.c, where interrupts are disabled and re-enabled while the ISR sets a button flag and resets the interrupt flag to allow subsequent interrupts.

Game Logic:

To create a sequence of flips for every game, a random number between 1 and 4 is chosen. This used the rand() function, which is preceded by a call to srand() in the orientation function, which is passed the milliseconds field of the real-time clock. This represents the number of flips in that game. Then, a random number between 0 and 1 is chosen for every flip. If the number is 1, then that flip is set to a flip about the z-axis (or a “flat” flip). If it is 0, then it is a y-axis flip. This flip sequence is then broadcast to the user via the LEDs: green for z-axis, blue for y-axis. All these flips are then scheduled as real-time processes with the same start times and deadlines (to ensure each is run one after the other) and process_select is then called to initiate the sequence. Throwing the board and returning it to its initial position ends a scheduled flip process and so the process then moves to the next one until all processes are completed.

Timing Diagram

[Fig. 7 Real-time Process Timing Diagram]

Process Queue

[Fig. 8 Process Queue]

The game consists of three main states: STATE_ORIENTATION, STATE_GAME_START, and STATE_GAME_END. During the orientation state, the player is tasked with orienting the board. After finding a suitable orientation position, the green LED turns on. The player presses the SW3 button to lock-in the position. This triggers an interrupt that sets the state to STATE_GAME_START, which generates a random sequence of flips for the player to perform. This sequence is delineated by yellow LED blinks.

State Machine Diagram

[Fig. 9: State Machine Diagram]

After every flip, a magenta LED will indicate that the toss was registered. The sequence is repeated for all scheduled flips. After completing the flips, the state is set to STATE_GAME_END where the player is notified of their performance through a sequence of green and red LED blinks. Red signifies that they failed that flip, while green signifies they successfully completed the flip. For instance, a sequence of red, green, red would tell the player that they failed the first and third flips, but succeeded on the second flip. This sequence repeats until the player presses the SW3 button again to change the state back to STATE_ORIENTATION.

The game keeps track of the flips using two integer arrays. Its length is 4, and it is initialized to negative-ones every game. One array holds the expected sequence of flips, with a 1 for z-axis flips and a 0 for y-axis flips. If the number of flips is less than 4, then the rest of the numbers in the array remain -1. During the actual game, the other array is set to 1, 0, or -1 if the flip recorded is a z-axis flip, y-axis flip, or failure, respectively. These two arrays are then compared at the end to determine whether to flash the red LED or green LED for each flip when giving the player feedback.

TESTING

The board was tested during development for each piece of functionality before integrating each piece into the final design. Final gameplay testing was performed using the full device, untethered, run through several game cycles, and verified that the RNG was working to generate pseudo-random sequences between game cycles. Free-fall detection was tested by performing a number of throws, as well as drop-tests, using an LED to indicate that the free-fall flag has been set. This was done simply by scheduling a process called LED_toss_catch() which turns the LED on when the flag is set for free-fall.

Flip detection testing was first performed with the device tethered to the computer, and in the hands. This meant that the flips could be tested in isolation from free-fall detection. We scheduled a process which checks to see if flips registered, called check_flip(), which was used to blink the LED corresponding to a detected flip. Performing a flip, then waiting, allows the LED to show if that flip was detected correctly. Flipping poorly, and then watching the LED, confirmed that “bad” flips were undetected. Failed tests for either of these conditions were then debugged by checking the data output over the tether via UART. The traces of false-positive or false-negative flips could then be further analyzed to see where the algorithm failed.

These pieces were then put together to perform one full flip. Flips were not detected unless they completed in mid-air, and the process completed if the throw ended (i.e. the board was thrown and caught). An LED showed which flip was detected when it was mid-air, and the red LED indicated that the throw “finished.” This confirmed that both free-fall and flip detection were operating well together, and could be implemented into the larger game-logic structure.

[Video Link 2: Unit Testing (Note: This video is purely supplementary material to show the testing process in more detail. Please do not count this toward our video timing length, as this is not our main video.)]

Failure Modes:

If the board is not calibrated correctly, good flips will not register, and bad flips might. This is a fundamental limitation of the magnetometer and our algorithm since we require that the game is static and oriented in the correct way before the game can begin.

If we had to do this project again in the future, with more time, and the ability to incorporate external hardware, we would have invested in a gyroscope in order to measure rotation accurately as an isolated phenomenon. Using numerical integration with a real-time clock, we would be able to estimate rotation as a signed measure in the full 360 degrees. Also, an LCD screen or dot matrix would be a benefit to the player, since using a single LED to ask for a set of flips is a very crude user-interface.

Another feature we would add is a time constraint on the player. By utilizing the deadlines of the real-time detect_flip processes, we would mark a flip attempt as "failed" if the player took to long to toss the board. We did not implement this feature because the idea had only occurred to us a few hours before the project deadline.

RESOURCES

The provided I2C project was instrumental in the completion of this project on-time. Sam Dipietro’s code base gave a great starting point from which we could develop “Flip It!” [1]. We made modifications to 3140_accel.c, as described above in the System Description section. The files gamify.c, test_mag.h, and test_mag.c were fully implemented by us from scratch. Of the files imported from Lab 5, process.c was made by us while realtime.h was provided by course staff for that lab. Our version of process.c had real-time-process functionality and periodic-real-time functionality (providing the current_time clock which we used to seed our random-number generator). “Flip-It!” requires multiple successive schedules of real-time processes to be generated and then run.

To develop this project, we referred to a number of sources on measuring orientation angles which have application in aerospace engineering, robotics, and many other disciplines. We also called upon our knowledge of physics to understand and analyze the data and the math behind some of the reference documents. The idea for checking zero-crossings came from [4].

  1. DiPietro, Samuel. “3140_accel” ECE 3140 Spring 2020 Github Repository: https://github.coecis.cornell.edu/ece3140-sp2020/3140_accel

  2. Harkins, Thomas E., Michael J. Wilson. “Measuring In-Flight Angular Motion With a Low-Cost Magnetometer.” Army Research Laboratory https://apps.dtic.mil/dtic/tr/fulltext/u2/a472265.pdf

  3. Kumar, Hemant. “Beginner’s Guide to IMU.” Robotics Club IITK. http://students.iitk.ac.in/roboclub/2017/12/21/Beginners-Guide-to-IMU.html

  4. Ruan, Tongjun, Robert Balch. “RPM measurement using 3-Axis Digital Magnetometer.” Int'l Conf. Embedded Systems, Cyber-physical Systems, & Applications. https://csce.ucmss.com/cr/books/2017/LFS/CSREA2017/ESC6055.pdf

  5. University of Illinois College of Engineering. Physics 101 Lecture 5. https://courses.physics.illinois.edu/phys101/sp2015/handouts/handout5.pdf

  6. Ashbaugh, M.S., Chicone, C.C. & Cushman, R.H. “The twisting tennis racket.” J Dyn Diff Equat 3, 67–85 (1991). https://doi.org/10.1007/BF01049489

  7. ThatsMaths.com. “The Intermediate Axis Theorem.” December 12th, 2019. https://thatsmaths.com/2019/12/12/the-intermediate-axis-theorem/

  8. NXP Semiconductors "FRDM-K64F Freedom Module User’s Guide." August 2016. https://www.mouser.com/catalog/specsheets/NXP_01112019_FRDM-K64F.pdf

  9. NXP Semiconductors "FXOS8700CQ 6-axis sensor with integrated linear accelerometer and magnetometer." 22 March 2016. https://media.digikey.com/pdf/Data%20Sheets/NXP%20PDFs/FXOS8700CQ.pdf

  10. NXP Semiconductors "K64 Sub-Family Reference Manual." 2 January 2014. https://www.mouser.com/datasheet/2/813/K64P144M120SF5RM-1074828.pdf

WORK DISTRIBUTION

We used Zoom’s remote control functionality to perform peer-programming for several parts of the project which were completed together. Organizing zoom meetings was quick and easy, but finding times where we were both available was very difficult during the finals period this semester. Below is a listing of the tasks completed by both, or each, of the group members. We dealt with issues by testing various functions to find errors or unexpected behaviors, talking through their possible causes, and then implementing and testing solutions. Whenever a new method was attempted together, the expected output was discussed before observing the actual output. This helped debug issues along the way.

There was a lot of miscommunication or lack of communication about availability throughout the project period due to the complications of end-of-semester activities including moving out of apartments and dorms, getting home, etc.

A rough outline of who was responsible for what:

Both:

Greg:

John: