Predict Lightning Strikes using Django and AWS

Predicting a lightning strike with 99.6% accuracy requires advanced algorithms, expert developers and data — a lot of data — so that the machine learning (ML) model can be trained and becomes reliable. FLASH pulls data from a variety of sources including weather radars and location-specific information like altitude, moisture, wind direction and temperature. It was imperative that FLASH develop an application that could pull, read and process the data both quickly and accurately.

Optimize the code structure

Working collaboratively with Will Howard, FLASH’s Lead Software Engineer, Six Feet Up developers reviewed and simplified FLASH’s existing Python code. Simplifying the code resulted in faster data processing speeds and code that was easy to both read and duplicate.

“The code is beautiful,” said Howard. “I was anticipating that it would take two weeks to do a multiple radar implementation, but thanks to your code updates, I was able to do it in just two days. I really appreciated working in the codebase after seeing the changes you made.”

Once optimized, Six Feet Up spread the code across the Django application and into AWS Lambdas.

Build an API with Django that is fast, efficient and accurate

FLASH needed an interface that could both request data from various web services and quickly make accurate predictions using FLASH’s proprietary and patent-issued algorithm. To achieve this goal, all of the Django application elements are stored in a single library, and all of the data is pulled directly from the cache. The algorithm is kept separate so that FLASH’s meteorologists can optimize the algorithm without disrupting the data collection.

A key element to the success of this project was validating and measuring the accuracy of the lightning strike predictions. During the development process, Six Feet Up pulled the existing code out of Django, pulled the existing code out of the Lambdas and put all of the data into a single package. This process guaranteed that the data would know how to talk to each other. At this stage, the developers were able to further simplify the code base, add additional context to the isolated code, and ensure that the data would load into cache keys. This process provided FLASH with the code it needed to run and rerun tests that validate the accuracy of the lightning strike predictions via varying locations, dates and weather conditions.

For example, FLASH states that Delta lost $125 million in revenue due to the grounding of its planes due to storm activity. The FLASH algorithm was run on archived radar data, and the airline would have saved tens of millions of dollars by indicating periods of diminished lightning threats utilizing the FLASH technology.

Deploy a product that is serverless and scalable

The Django application integrates seamlessly with the serverless components of AWS including AWS Lambda and AWS Fargate.

“No human will need to go into the application to patch and/or update an OS, on a server, at any time,” said Calvin Hendryx-Parker, Six Feet Up’s CTO. “This is a huge win. FLASH’s application is built to scale. It won’t need high-touch maintenance because we built it to be serverless from the beginning.”

Finally, the FLASH application is deployed with GitLab CI/CD merge request pipeline.

Watch "Predicting Lightning Strikes with AWS and Python":

FLASH now has a working product that generates accurate lightning strike predictions with 99.6% accuracy. As a result of Six Feet Up’s work, the application — which initially took minutes to make predictions — can predict when and where lightning will strike in just seconds.

“The foundations laid with the model’s packaging and integrability into the rest of the system illustrates the level of skill and forward-thinking by the team at Six Feet Up,” said Howard.