The Learning continues into 2021

The start of 2021 saw us road tripping into Queensland to service the units we installed in the back end of 2020.

Trip 2, Lesson 1: New components force some modifications on the go!

Winning the ICON Grant September 2020 (thanks CBRIN and the ACT Government) meant we had to somewhat rapidly expand out FatigueM8 fleet from 5 to 10 units. Our FatiguM8’s are made up of several “off the shelve” components and as with all technologies every few months there are new versions released, updates, enhancements and as it happens subtle yet important layout changes.

The process we follow to assemble the FatigueM8 compute units is well refined now, after building over 23 prototypes now. The process starts with the installation of the operating system and configuring the various options; next the specific software components are installed and finally the FatigueM8 code base. We then run through a series of connection tests for the ECG unit, GPS and testing the LED lights. When all the tests pass, we assemble the FatigueM8 unit.

It was at the point of assembly that we noticed a subtle, yet important change in the computers layout. In the latest model, the USB and Network ports have been switched (no idea why!). We us the USB port to connect a 3G/4G modem to enable almost realtime up load of data for analysis; and remote connection.

Before (left) and After (right) pictures of FatigueM8 front plate.

Fortunately our prototypes are exactly that, prototypes and allow for quick tweaks without major cost implications. As was the case in Lesson 1 above, a hotel room was able to be converted to a mini-workshop and it was Bunnings to the rescue this time. Grabbing a small push saw and rasp, 3 minutes later we’d extended the USB whole to account for the layout switch. We’ll incorporate this new design into the next FatigueM8 print run.

Trip 2, Lesson 2: What happens when the backup, backups’ fails?

The starting premise of FatigueM8’s steering wheel installation is that the drivers shouldn’t have to do anything other than drive to use the system. To facilitate this we have the ECG wired into the truck’s electrical system, with a battery that is charged when the trucks lights are on. FatigueM8 uses the battery power when then the vehicles lights aren’t on and/or the truck is turned off. The battery is able to power the ECG unit for roughly 5 days and during the year this has worked seamlessly with our trial trucks.

Coming back from the 2020 Christmas break several of our trucks had been off the road for 2 weeks (or there a bouts) and the FatigueM8 systems came back online but had no ECG unit connection. After several days of debugging and speaking to the ECG Unit manufacturer we discovered there is another tiny battery inside the ECG unit itself, which provides power to the internal clock. We discovered that the life of the clock battery is about 5 days and when it goes flat if puts the ECG unit into a “safe state” that requires a hard reset. We’ll thank 2020 for the year that kept on giving for this one 🙂

Trip 2, Lesson 3: Securing the dashboard unit needs some work!

Our FatigueM8 dashboard unit (which contains the compute unit) also has a forward facing camera that we use to capture the road/driving conditions our drivers are operating in; and is designed to sit on the trucks dashboard. This sounds simple enough, but as it turns out there isn’t much consistency in dashboard layouts; even within the same brand and model of truck. Recently I went to check in on a couple of our installed units and found one upside-down! In this instance we’d underestimated the amount of road vibrations coming into the cab and this truck travels several times a day along a dirt road for 50kms.

We’re swapped the unit from the drivers side over to the passenger side of the truck and used a humble cable tie to secure the unit to the air vent and hopefully in place! We’ll check back in a month or so.

When we installed our units wee used a small “occy strap” to secure the unit, but it appears we need a little extra securing. Another of our trial units used Velcro to stick the unit in place, which seams to work well.

We’ll be working on the Dashboard unit over the coming iterations with a focus on securing the unit, as well as making the design of the unit more adaptable to different dashboard configurations.

Until next time, stay safe.

Lessons from the late 2020 FatigueM8 installations

Trip 1, Lesson 1: There isn’t many things that K-mart isn’t able to fix!!

In my preparations for the trip to FNQ we had built three (3) new FatigueM8 compute units, taking our total builds at that point in time to fourteen. As part of the pre-installation checks I noticed that one of the software components hadn’t been configured. Real VNC isn’t part of our core stack and doesn’t effect the operation and collection of thee ECG, but it is critical for accessing the remote device for debugging. When connected to our network at build time, configuring Real VNC is easy; just establish an ssh session, connect to the device and enable it and then connect using thee Real VNC viewer, job done. However, when outside the office and connected to Telstra’s 3G/4G network it’s a little harder. 

After scratching my head for couple of hours, and thinking a little outside of the box; I remembered that the compute unit has USB and HDMI ports, which means it can be setup as a computer. The only issue, I didn’t have a spare Keyboard, Mouse or Monitor in my carry on luggage. Enter K-Mart. K-Mart stocks a small range of computer peripherals and I was in luck, K-Mart Mount Isa to the recuse. With a new keyboard, mouse and HDMI capable I headed back to the hotel; Using the HDMI port on the television I was able to turn the FatigueM8 compute unit into a mini computer, login and configure Real VNC to enable remote connections. Problem Solved. 

FatigueM8 converted into a computer, thanks to K-Mart!

In only a weeks’ time this effort to setup remote debugging would prove worth every moment of effort to setup (more on that later).

Frasers Yarn in the Yard 2020

 The team at Frasers Livestock transport have been long time supporters of our FatigueM8 journey, with several of the team present at the FatigueHACK in 2018 where the solution was born. They were amongst the first, if not the first, to volunteer to be trial partners of our emerging technology.

On Friday last week (18th December 2020) the team at Frasers held their annual “Yarn in the Yard” driver health day and when Athol asked if we could be part of the session we jumped at the chance. The line up of presenters had us experience some serious imposter syndrome; Dr Ross and Dr Heart truck founder, as well as The Assistant Minister for Roads and the CEO of the HNVR to name but a few.

Frasers main depot is located in Warwick QLD in the heart of QLD’s Agriculture belt (?) and we were in awe of the amount of impeccably maintained equipment that was in the yard when we arrived. We were part of the “Fatigue & Distraction” session sharing the stage with Andreas (NHVR Fatigue expert), Dr Darren Wishart (Griffith University) and Sgt Paul Kelly (Commander Heavy Vehicle Operations QLD).

Heart of Australia at Frasers Yarn in the Yard

We presented a short summary of our journey to date and also participated in an interactive panel discussion with the attendee’s.

Our presentation had two (2) purposes, firstly to explain how FatigueM8 is aiming to tackle fatigue and secondly to recruit a volunteer to have a FatigueM8 unit installed. We were able execute both parts of the plan, sharing the story and getting a volunteer (well several actually). The candidate truck was a T659 (Frasers only have the one model in their fleet) which had joined the fleet only a couple of months beforehand. 

We’re getting faster and faster with the installation process, this time around it took roughly one and a half hours, which included a couple of chats with curious team members who stopped for chat when they saw me in the cab. 

The Yarn in the Yard session was the catalyst for a couple of other side visits for the FatigueM8 team. On the way back from Warwick to Brisbane (where I’d be flying home from) we stopped into see the folks at TyTec Logistics. Tytec runs a fleet of K200’s transporting oversized tyres to which support mining operations throughout Australia.

One of Tytecs’ K200 fully loaded. (image sourced from google)

Interestingly when we jumped into the cab of the impressive K200 the driver already had a steering wheel cover on (see below left) and it was the same model that we’d used many times in our earlier prototyping and trial activities (below right). 


Having completed the Frasers installation only hours before hand, we knocked this installation over in an hour and 15 mins (which included 10 mins to grab the notebook out of the car). 

Both the Frasers and Tytec trucks hit the road Sunday and the data started flowing into the system. Below are a couple of pictures from the forward facing camera recording the road conditions.

Until next time, stay safe

Far North Queensland Part 2 – A quad-trailer road train for install #4

After the humidity of Townsville, the dry heat of Mount Isa’s 36 degrees was a welcome change. We arrived in Mount Isa early ready to undertake our fourth FatigueM8 installation, rather fittingly as it happens into a massive Quad Trailer road train, operated as one of 50 by Wagner’s Mount Isa operation that was our 4th installation.

These units really are road trains, with a length well over 50 Meters and weighting north of 120 tonnes, these beasts run 24 hours a day, 7 days a week.

Noting that Mount Isa is at least a five (5) hour plane trip from our base in Canberra and leveraging our software quality assurance heritage after landing, we went through a final set of tests before heading out to site to complete the installation. A couple of tweaks and we were on our way.

Once on-site the installation process is pretty straight forward and took just shy of two (2) hours to complete. The end result is in one way, quite underwhelming, as the steering wheel cover just blends into the wheel and part from the coloured zip it’s hardly noticeable. The zip is a recent addition to the steering wheel cover and simplifies the installation (whilst not annoying the driver). The inclusion of the zip takes roughly an hour out of the installation time. We are interested to see how the zip based unit performs and what our drivers think of it, so watch this space.

The compute unit also blends in somewhat. The dashboard of the T659’s is quite a bit steer than the other trucks we’ve installed in and the result is that the compute unit is somewhat hidden. This isn’t a bad thing, in our original design back in 2018 we’d envisaged the compute unit being a black box hidden somewhere in the truck. Luckily the design of our front camera module is such that it was able to be adjusted to the perfect angle (thanks to our previous trial feedback). 


The truck we installed the unit into was in for a service and I was relieved to see data starting to flow from the unit the next morning (albeit from the truck moving around the yard). This truck, and all the other rigs in Mount Isa run in a 24 hour operation, so we’re excited to see more data flowing in once it hits the road post service. 

Next week we’re off to QLD installing more units, so be sure to check in again to see where we’ve been.

 
Stay save until next time.

Far North Queensland Trip – Part 1 Townsville (presentation time).

This week’s FatigueM8 Friday comes to you following a trip to far North Queensland. Augmented-Intelligence (makers of FatigueM8) were part of a event held at the Smart Precinit  North Queensland (SPNQ) created by our partners KJR Australia. The event titled “Event: How AI is Advancing Workplace Health and Safety” saw presenters from James Cook University, Sparta Science, Cernova  Neuroscience and Augmented-Intelligence share their insights and research findings related to Artificial Intelligence (AI) and Machine Learning (ML).

The performance of athletes and employees have more physical indicators in common than we might expect. So how can technology that has traditionally been used to monitor athlete health and performance be transferred to employees across a range of industries to ensure their health and safety?

The Event in session

The event was created to coincide with Far North Queensland hosing the 2020 edition of the Womens National Basketball League (WNBL). KJR Australia is a proud sponsor of the University of Canberra Capitals (UC Caps) and managed to find a four (4) day gap in their hectic schedule to co-host the event.


Our co-founder Andrew had dual hat’s on for the day, representing both KJR Australia and Augmented-Intelligence. The event gained significant media attention with the ABC interviewing Dr, Richard Franklin from James Cook University and WIN News interviewing Andrew. The news story aired on the nightly news around Queensland.

The full event recording will be available soon (watch this space).

You maybe wondering how FatigueM8 is connected with the AI and workplace health and safety? Well, our FatigueM8 system uses Artificial Intelligence (AI) and Machine Learning (ML) to analyse the Electrocardiograph (ECG) observations recorded from our drivers. After collecting the samples, which are continually recored when ever the driver has two hands on the steering wheel, looking for signs of Fatigue firstly and secondly looking for signs of other cardiovascular disease or abnormalities. The ECG equipment we use at the core of our system is hospital grade, FDA approved ECG and that allows for detailed analysis of the samples. After the installation of the FatigueM8 system we set the system into Baseline mode and observe the drivers. Once the baseline is created, we’ll then start to assess the drivers physiological state and actively alert if we detect, or predict a fatigue event based on the ECG observations.

Watch out for the next installment where we head from Townsville 900kms inland to Mount Isa to commission our 4th FatigueM8 unit, quite fittingly into a Quad Trailer road train! 

Until next time, stay safe.

Innovation Grant success, onwards and upwards!

ACT Government and CBR Innovation logos

It’s offical now, Augmented-Intelligence was awarded one of the ACT Government’s Innovation Grants (ICON) as part of the most recent round. The Grant will be used to further our product trials and build out the FatigueM8 portal.

The ICON Grant, along with matched contributions from Augmented-Intelligence and our partners will see close to $100k invested into our project “Fatigue and cardiovascular monitoring for heavy vehicle drivers using an ECG monitor embedded in the steering wheel.”.

The ICON Grants rounds are always very competitive and we are ever so thankful to the the ACT Government and CBR Innovation for support of our journey. We’ll be putting the collective funds to good use, purchasing the hardware to create an additional five (5) FatigueM8 units. The additional units will allow our current trial plans to be doubled from five (5) to ten (10) vehicles, with potential for up to 20 drivers to participate.

As well as the additional units, we’ll also be creating a FatigueM8 portal, where our drivers will be able to login and review their data in an easy to use manner. Watch this space for further developments.

You can read the official press release here

FatigueM8 Friday – the next chapter begins

This week we welcome into the FatigueM8 team, Katie Speer, who’s a Ph. D Candidate at the University of Canberra (UC). Katie is studying within the Department of Sport & Exercise Science, Faculty of Health and has published several research papers investigating Heart Rate Variability (HRV) in connection with the autonomic nervous system, maternal exercise influence on childhood HRV and also investigated the ability of commercially available devices to measure HRV in children. All of her papers are available via the link below.

https://www.researchgate.net/profile/Kathryn_Speer

Katie joins the FatigueM8 team for a five (5) month internship to undertake a research project titled “The Validation of the FatigueM8 System for using heart rate variability to detect truck driver fatigue”. Katie will be supported by UC Associate Professor Andrew Mc Kune. The FatigueM8 team was introduced to Professor Mc Kune as we researched the potential cause of HRV observations we’d captured after a trip to Threadbo mountain biking.

Read that post here (http://www.augmented-intelligence.com.au/wp-admin/post.php?post=181)

Professor Mc Kune was a review on the paper we referenced here (doi: 10.3389/fphys.2017.00301) and noting he was a local Canberrian we reached out.

Professor Mc Kune’s “interests include investigating bioenergetic and stress responses (inflammation, autonomic nervous system, HPA-axis) in chronic disease. How do exercise and/or nutrition interventions impact these systems to prevent and treat chronic disease? Methodologies include heart rate variability, salivary bioscience, cardiopulmonary exercise testing, telomere length and ultrasound.”

We’re super excited to have Katie and Andrew onboard; as part of the project they’ll be undertaking we’re going to be expanding our in-cab trials and data collection.

Watch this space for some exciting announcements over the coming weeks.

Image classification, looking forward (and sideways)

A couple of Friday’s have past since the last post, but it’s not because of a lack of activity on our side. We been busily working away to grow out our capability. One of the initial features that we had in mind from the outset of FatigueM8, was to add context to the ECG observations by overlaying additional information sets. This week we’ve brought online an additional data stream, leveraging AWS Rekognition service.

As we have explored in previous posts, we’ve had our fair share of challenges with our forward facing camera. Adding in the Rekognition classifications threw up a couple more curve balls to us. The pictures below are from two (2) of our trucks were the FatigueM8 unit had shifted since being installed. Being prototypes we’d used “suckers” to connect the unit to the wind screen and with the changing of temperatures the suction was lost resulting the cameras pointing every which way.

A quick trip to the local hardware store has us back up and pointing in the right direction! A little 25 cms elastic strap and a couple of cable ties have the install back on track.

The resulting photos (below) are back to being captured correctly and ready for processing through rekognitition. Now when an image is uploaded from the truck into the cloud it’s classified by rekognition and then depending on what’s been found, boxes are drawn around the objects.

If a person is detected in the image, we’ll blur them out or don’t use the image just to be safe.

As we run all the images through this process we’ll be able to build up a picture (no pun intended) about the traffic, road and environmental conditions our drivers are driving in; and this will add more context to the ECG observations we collecting.

Until next time, stay safe.

FatigueM8 Friday – start ups and downs

To kick off this week’s edition we start with a nice time series set of images captured from one of our units based in Canberra. On Thursday morning Canberra awoke to a thunderstorm and in this set of images (which span 7 minutes) you can see the rain falling and then freezing on the bonnet of the truck.

This week saw the commissioning of a new FatigueM8 unit, into one of Multiquip’s Kenworth T408 SAR’s, which is setup pulling a B-Double between Marulan and several locations in Sydney.

Multiquip Kenworth 408 SAR, with FatigueM8 installed.

It’s the first installation into a T408, which is an older model Kenworth (circa 2009/10), fortunately we’d been working with steering wheel in our lab which was the same model as is installed in the T408.

The installation was straight forward and with the steering wheel cover installed powered it up for testing. We observed the ECG monitor “cycling” through the start up sequence over and over again. When measured the electrical current coming through the steering column we noticed some variance in the electrical current. The voltage appears to be bouncing between 11.5 volts and up to 14.5 volts; our ECG monitor operates up to 13.2 volts before switching off to protect the electrical circuit.

In the spirit of rapid prototyping, we setup a voltage regulator in the lab, with the aim of providing a stable 12 volts to the ECG unit. We assembled a linear voltage regulator (below) which was able to drop the power by at least ~1 volt.

We attached the regulator to the T408’s horn cover (pictured below) and set about testing our theory.

Voltage regulator prototype installed into the T408

We had some success with the voltage regulator in place, the ECG unit powered up and connected to the FatigueM8 on-board unit. We were able to capture the ECG signal for a period of time, before it shutdown and disconnected. We tested all the connections and power supplies and all appeared/reported working at the expected voltage. Our investigations are continuing, watch this space for the next update!

On the positive side, we’re starting to collect data from the front camera and GPS which will be fed into our models.

Until next time, stay safe.

FatigueM8 Friday’s – when you’re born in the cloud, is the sky the limit?

On Wednesday this week I had the opportunity to present our Augmented-Intelligence story to the AWS Canberra User group (Amazon Web Services). It was great chance to reflect on our journey thus far, revisiting the promises and commitments we made to the Australian Trucking community at the 2018 FatigueHACK.

From the beginning, leveraging the cloud (and AWS) were front of mind. As I flicked through the official photos from the event I came across this one (below) from about midway through the first day. AWS Greengrass and AWS Sagemaker are called out as part of the solution; as it happens both of these products are still in the roadmap, but not yet implemented.

Augmented-Intelligence – initial architecture thoughts April 2018.

But that’s enough of the past, let’s look at what we have currently implemented. The picture below appeared in our presentation and I’ve marked up what we have currently implemented and being uploaded into AWS in the cloud.

We have three (3) FatigueM8 units installed and commissioned, and in each we are capturing the drivers ECG observations, the view out the front wind screen and geocoding each of the observations. From these we’re able to derive vehicle speed, distance travelled, driving time, road conditions, traffic conditions, drivers Heart Rate and a whole host of other statistics.

In my presentation from Wednesday night drilled into all of these areas, highlighting the how we leverage the various AWS Services (AWS S3, AWS Lambda, AWS Anthea and AWS QuickSight) to capture, store, analyse and present the data that we feed into our models to calculate the FatigueM8 Fatigue Score (aka FFS).

Wednesday’s presentation was recorded and is now available to be watched (or re-watched) via the following link. The FatigueM8 overview starts about 17 mins into the recording. You can check out the presentation here https://www.twitch.tv/videos/659965587

Looking back, we promised to spend the prize money on research (and not beer) which we’ve done; oh boy did we stretch that $6k a loooonnnnngggg way. We committed to continue to repay generosity shown to us during the competition, provide objective data (eg ECG recordings) to contribute to a better understanding of how fatigue effects individuals and to help everyone “keep on trucking” in a healthy and sustainable manner.

As for the sky being the limit, well being born in the cloud does have it’s advantages, but its important that we remain focused on supporting those who’s hands are on the steering wheel and feet firmly on the pedals!

Until next week, stay safe.