FirmWareX

• 

  Landing page

Inspiration

Modern devices - from routers and industrial controllers to connected infrastructure - depend on firmware that may contain cryptographic implementations with security implications. Yet, when source code is unavailable, understanding what cryptographic algorithms are present and where further investigation is needed can require extensive reverse engineering.

We started FirmWareX with a simple question: Can machine learning help turn firmware analysis from a time-consuming manual process into a faster, evidence-driven security workflow?

The motivation goes beyond detecting a vulnerability. Organizations also need to understand their cryptographic attack surface, identify potentially weak implementations, and prioritize security assessments before compromised devices become a larger problem.

What it does

FirmWareX is a firmware security analysis platform that aims to help identify potential vulnerabilities and weak or outdated cryptographic implementations in compiled firmware.

Its intended capabilities include:

• Firmware analysis: Accept firmware images for automated inspection.
• Cryptographic classification: Use machine learning to classify patterns associated with cryptographic implementations.
• Security assessment: Help identify findings that warrant further investigation.
• Actionable reporting: Present analysis results in a format that helps security teams understand and prioritize potential risks.
• A path toward scalable assessment: Reduce the manual effort involved in analyzing firmware across different devices and environments.

The long-term vision is to help government institutions, device manufacturers, and private-sector security teams better understand the cryptographic components embedded in their firmware.

How we built it

We developed a web-based application with a frontend for user interaction, a backend API for handling uploads, and a machine learning prototype for cryptographic classification.

Our current classifier uses XGBoost, a gradient-boosted decision-tree algorithm, trained on engineered numerical features extracted from labelled cryptographic implementation samples. The prototype covers 10 classes, including AES, RSA, ECC, SHA, DES, and other cryptographic or related categories.

The XGBoost objective can be expressed as:

$$ \mathcal{L}(\phi)=\sum_{i=1}^{n}l(y_{i},\hat{y}{i})+\sum{k=1}^{K}\Omega(f_{k}) $$

Here, (l) measures prediction error, while (\Omega) penalizes model complexity. This helps balance predictive performance with regularization.

For security analysis, information theory also provides a useful foundation. Shannon entropy measures the uncertainty in a distribution:

$$ H(X)=-\sum_{i=1}^{m}p(x_{i})\log_{2}(p(x_{i})) $$

Entropy can help flag regions of a binary that exhibit unusual statistical characteristics. However, high entropy does not prove encryption or a vulnerability; it can also arise from compression or other forms of encoded data.

We evaluate the classifier using held-out data rather than relying solely on training performance. Accuracy is defined as:

$$ \text{Accuracy}=\frac{N_{\text{correct}}}{N_{\text{total}}}\times 100\% $$

Precision measures how many positive predictions are correct:

$$ \text{Precision}=\frac{TP}{TP+FP} $$

Recall measures how many actual positive samples are identified:

$$ \text{Recall}=\frac{TP}{TP+FN} $$

The F1-score is the harmonic mean of precision and recall:

$$ F_{1}=2\frac{\text{Precision}\times\text{Recall}}{\text{Precision}+\text{Recall}} $$

Our current controlled-dataset results are:

Metric	Result
Training accuracy	99.07%
Validation accuracy	94.59%
Test accuracy	95.14%
Classification classes	10

These results establish an encouraging baseline for the classification task. They are not a claim of 95.14% accuracy on arbitrary real-world firmware; validating that broader use case is part of our ongoing work.

Challenges we ran into

One of the central challenges is that firmware is compiled, architecture-dependent, and often stripped of information that would be available in source code. The same cryptographic algorithm can produce different binary patterns depending on its compiler, optimization settings, target processor, and implementation.

Other challenges include:

• Distinguishing cryptographic implementations from unrelated code with superficially similar characteristics.
• Working with imbalanced datasets in which some classes have substantially fewer examples than others.
• Avoiding overreliance on aggregate accuracy when individual classes may have different error rates.
• Separating statistical signals from reliable evidence of a security weakness.
• Designing an automated workflow without treating an ML prediction as definitive proof of a vulnerability.

A particularly important engineering challenge is ensuring that the features used during training correspond to those available during actual firmware analysis. A model that performs well on one representation cannot automatically be assumed to generalize to another.

Accomplishments that we're proud of

We have built an initial working application, implemented the firmware upload workflow, and developed a machine learning classifier with a 95.14% test accuracy on a controlled dataset.

Beyond the numerical result, we have established a foundation for connecting firmware analysis with ML-assisted cryptographic classification. This gives us a concrete starting point for moving from an isolated classification experiment toward a practical security assessment workflow.

We are also proud of approaching the problem with an emphasis on measurable performance and technical limitations rather than presenting model predictions as guaranteed vulnerability detections.

Our goal is not merely to produce a label. It is to build a system whose findings can eventually be investigated, explained, and used to make better security decisions.

What we learned

We learned that a strong ML result is only one part of a reliable cybersecurity product.

Dataset quality, class imbalance, feature consistency, generalization, and the ability to validate findings all matter. We also learned that cryptographic identification and vulnerability detection are related but distinct tasks: recognizing a cryptographic implementation does not automatically establish that it is insecure.

This shaped our approach toward evidence-based reporting, careful evaluation, and a gradual transition from controlled experiments to realistic firmware samples.

We now see FirmWareX not simply as a classifier, but as the foundation for a broader firmware security analysis workflow.

What's next for FirmWareX

Our next priority is to validate the approach against real firmware binaries and broaden support for different processor architectures, compilers, and optimization settings.

We plan to work toward:

• Integrating automated extraction and reverse-engineering tools into a unified analysis pipeline.
• Training and evaluating models on binary-level evidence that matches the actual inference workflow.
• Improving class-wise evaluation and reducing false positives.
• Connecting classifications to evidence that analysts can inspect and verify.
• Developing risk-oriented reports that distinguish observed evidence, model predictions, and confirmed security findings.
• Exploring deployment options suitable for organizations with strict data-security and infrastructure requirements.

Our long-term vision is to make firmware cryptographic intelligence faster, more explainable, and more accessible - helping organizations understand what is inside their devices before those devices become a security liability.

Built With

• ai
• binwalk
• cybersecurity
• fastapi
• ghidra
• machine-learning
• postgresql
• python
• react
• redis
• xgboost