Inspiration
Our healthcare system has a problem: it’s focused on treating diseases years after they start, which jeopardizes patient survival and costs thousands of dollars in treatment plans. Genetic data analysis only provides vague risk levels, and not data about any conditions the patient may currently have. Biomarker data is more accurate for current conditions, but assaying hundreds of biomarkers to get a complete picture of patient health is expensive, time-consuming, and inefficient. Epigenetic data is the necessary linker between genes and the environment. Using publicly available epigenetic data, such as RNA seq, methylation levels, and chromatin mapping, we aim to build an AI that can give a full picture into patient health and identify health threats before symptoms even start.
What it does
Our system shows a heatmap of genes differentially upregulated in different types of cancer, and includes a window for users to upload their own patient data and predict if the cells are cancer and if so, what type. We also have a tab that allows users to upload larger datafiles to train the internal model themselves. In the biological age predictor tab, you can upload patient methylation or expression data and use the predictor to determine their biological age and risk factors for different age-related diseases. You can also use the data uploaded to generate your own elastic neural model with the learning settings of your choice, and export it for personal use.
How we built it
First, datasets were procured from kaggle, GEO, and TCGA in the form of .csv files. These datasets were normalized using a script edited by Amazon Q in vscode to prevent any repeats or errors from contaminating AI learning. Then, an optimized neural network was trained in vscode to cancer patient data and exported as a .pth file. The code is listed below, since it was not used in the final application. Next, we uploaded all data files and trained model to replit, which generated a UI around the instructions.
Challenges we ran into
Many files had nonstandard gene name formats (for instance, some used ENSG while others used HUGO format). For this, we asked an AI to manually comb through each row and search online for the HUGO equivalent to each other format, since HUGO is more easily understood by biologists. The UI also had some issues with uploading and analyzing files properly, which were eventually fixed after some trial and error.
Accomplishments that we're proud of
Neither of us have any coding experience before--we didn’t know what pytorch even was before this event, much less how to build a UI with functional elastic model training. The technology of today has allowed us to turn an idea into a feasible reality in less than two full days, which is incredible to see. We are super proud to even have a functioning product in the first place, much less that it displays accurate data.
What we learned
Many Dev skills, including vscode, Amazon Q, CNN training, pytorch library usage, replit, and more.
What's next for WeeklyPulse
We plan on uploading more diverse forms of data so that the application can not only predict cancer status and biological age, but any other relevant disease condition that the patient might have.
from sklearn.preprocessing import LabelEncoder
def load_and_preprocess_data(filepath: str): """ Load and preprocess the data with detailed error checking. """ try: # Load data print(f"Loading data from {filepath}") df = pd.read_csv(filepath) print(f"Data shape: {df.shape}")
# Print column information
print("\nColumns in dataset:")
for col in df.columns:
print(f"- {col}")
# Extract features (all columns except labels)
feature_cols = [c for c in df.columns if c not in ["healthy", "cancer_type"]]
print(f"\nNumber of feature columns: {len(feature_cols)}")
# Handle categorical columns by encoding them
for col in feature_cols:
if df[col].dtype == 'object': # If the column is categorical
print(f"Encoding categorical column: {col}")
encoder = LabelEncoder()
df[col] = encoder.fit_transform(df[col])
# Convert features to numpy array
X = df[feature_cols].values.astype(np.float32)
print(f"Feature array shape: {X.shape}")
# Extract labels
y_health = df["healthy"].values.astype(int)
y_cancer = df["cancer_type"].values.astype(int)
print(f"Health labels shape: {y_health.shape}")
print(f"Cancer labels shape: {y_cancer.shape}")
# Print unique values
print(f"\nUnique health values: {np.unique(y_health)}")
print(f"Unique cancer types: {np.unique(y_cancer)}")
return X, y_health, y_cancer
except Exception as e:
print(f"Error in data loading: {str(e)}")
raise
import torch import torch.nn as nn import torch.optim as optim from torch.utils.data import TensorDataset, DataLoader import pandas as pd import numpy as np from sklearn.model_selection import train_test_split from sklearn.preprocessing import LabelEncoder
def load_and_preprocess_data(filepath: str): """ Load and preprocess the data with detailed error checking. """ try: # Load data print(f"Loading data from {filepath}") df = pd.read_csv(filepath) print(f"Data shape: {df.shape}")
# Print column information
print("\nColumns in dataset:")
for col in df.columns:
print(f"- {col}")
# Extract features (all columns except labels)
feature_cols = [c for c in df.columns if c not in ["healthy", "cancer_type"]]
print(f"\nNumber of feature columns: {len(feature_cols)}")
# Convert features to numpy array
X = df[feature_cols].values.astype(np.float32)
print(f"Feature array shape: {X.shape}")
# Extract labels
y_health = df["healthy"].values.astype(int)
y_cancer = df["cancer_type"].values.astype(int)
print(f"Health labels shape: {y_health.shape}")
print(f"Cancer labels shape: {y_cancer.shape}")
# Print unique values
print(f"\nUnique health values: {np.unique(y_health)}")
print(f"Unique cancer types: {np.unique(y_cancer)}")
return X, y_health, y_cancer
except Exception as e:
print(f"Error in data loading: {str(e)}")
raise
class DualCNN(nn.Module): def init(self, input_size, num_cancer_types): super().init()
# Print network configuration
print(f"\nInitializing network with:")
print(f"Input size: {input_size}")
print(f"Number of cancer types: {num_cancer_types}")
# Convolutional layers
self.conv_layers = nn.Sequential(
nn.Conv1d(1, 32, kernel_size=3, padding=1),
nn.BatchNorm1d(32),
nn.ReLU(),
nn.Dropout(0.3),
nn.Conv1d(32, 64, kernel_size=3, padding=1),
nn.BatchNorm1d(64),
nn.ReLU(),
nn.Dropout(0.3)
)
# Calculate the size after convolutions
self.flat_size = 64 * input_size
# Fully connected layers
self.fc_shared = nn.Sequential(
nn.Linear(self.flat_size, 256),
nn.ReLU(),
nn.Dropout(0.5),
nn.Linear(256, 128),
nn.ReLU(),
nn.Dropout(0.5)
)
# Output heads
self.health_head = nn.Linear(128, 1) # Binary classification
self.cancer_head = nn.Linear(128, num_cancer_types) # Cancer type classification
def forward(self, x):
# Add channel dimension
x = x.unsqueeze(1) # Shape: (batch_size, 1, features)
# Convolutional layers
x = self.conv_layers(x)
# Flatten
x = x.view(x.size(0), -1)
# Shared fully connected layers
x = self.fc_shared(x)
# Output heads
health_out = self.health_head(x).squeeze(1)
cancer_out = self.cancer_head(x)
return health_out, cancer_out
def train_model(model, train_loader, val_loader, device, num_epochs=50): """ Train the model with detailed progress tracking. """ print("\nStarting training...") print(f"Using device: {device}")
# Loss functions
criterion_health = nn.BCEWithLogitsLoss()
criterion_cancer = nn.CrossEntropyLoss()
# Optimizer
optimizer = optim.AdamW(model.parameters(), lr=0.001, weight_decay=0.01)
# Training loop
best_val_loss = float('inf')
patience = 10
patience_counter = 0
for epoch in range(num_epochs):
# Training phase
model.train()
train_loss = 0
health_correct = 0
cancer_correct = 0
total = 0
for batch_idx, (data, health_labels, cancer_labels) in enumerate(train_loader):
try:
# Move to device
data = data.to(device)
health_labels = health_labels.to(device)
cancer_labels = cancer_labels.to(device)
# Forward pass
health_out, cancer_out = model(data)
# Calculate losses
loss_health = criterion_health(health_out, health_labels.float())
loss_cancer = criterion_cancer(cancer_out, cancer_labels)
loss = loss_health + loss_cancer
# Backward pass
optimizer.zero_grad()
loss.backward()
optimizer.step()
# Calculate accuracy
health_pred = (torch.sigmoid(health_out) > 0.5).long()
cancer_pred = cancer_out.argmax(dim=1)
health_correct += (health_pred == health_labels).sum().item()
cancer_correct += (cancer_pred == cancer_labels).sum().item()
total += health_labels.size(0)
train_loss += loss.item()
if batch_idx % 10 == 0:
print(f'Epoch: {epoch}, Batch: {batch_idx}, Loss: {loss.item():.4f}')
except Exception as e:
print(f"Error in training batch {batch_idx}: {str(e)}")
print(f"Batch shapes - Data: {data.shape}, Health: {health_labels.shape}, Cancer: {cancer_labels.shape}")
raise
# Calculate epoch metrics
train_loss = train_loss / len(train_loader)
health_acc = 100. * health_correct / total
cancer_acc = 100. * cancer_correct / total
# Validation phase
model.eval()
val_loss = 0
val_health_correct = 0
val_cancer_correct = 0
val_total = 0
with torch.no_grad():
for data, health_labels, cancer_labels in val_loader:
data = data.to(device)
health_labels = health_labels.to(device)
cancer_labels = cancer_labels.to(device)
health_out, cancer_out = model(data)
loss_health = criterion_health(health_out, health_labels.float())
loss_cancer = criterion_cancer(cancer_out, cancer_labels)
loss = loss_health + loss_cancer
val_loss += loss.item()
health_pred = (torch.sigmoid(health_out) > 0.5).long()
cancer_pred = cancer_out.argmax(dim=1)
val_health_correct += (health_pred == health_labels).sum().item()
val_cancer_correct += (cancer_pred == cancer_labels).sum().item()
val_total += health_labels.size(0)
# Calculate validation metrics
val_loss = val_loss / len(val_loader)
val_health_acc = 100. * val_health_correct / val_total
val_cancer_acc = 100. * val_cancer_correct / val_total
# Print epoch results
print(f'\nEpoch {epoch}:')
print(f'Train Loss: {train_loss:.4f}, Health Acc: {health_acc:.2f}%, Cancer Acc: {cancer_acc:.2f}%')
print(f'Val Loss: {val_loss:.4f}, Health Acc: {val_health_acc:.2f}%, Cancer Acc: {val_cancer_acc:.2f}%')
# Early stopping
if val_loss < best_val_loss:
best_val_loss = val_loss
torch.save(model.state_dict(), 'best_model.pth')
patience_counter = 0
else:
patience_counter += 1
if patience_counter >= patience:
print("Early stopping triggered")
break
def main(): # Configuration filepath = "dual_output_training_data.csv" # Update with your file path batch_size = 32 device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
try:
# Load and preprocess data
X, y_health, y_cancer = load_and_preprocess_data(filepath)
# Split data
X_train, X_val, yh_train, yh_val, yc_train, yc_val = train_test_split(
X, y_health, y_cancer, test_size=0.2, random_state=42, stratify=y_health
)
# Create data loaders
train_dataset = TensorDataset(
torch.FloatTensor(X_train),
torch.LongTensor(yh_train),
torch.LongTensor(yc_train)
)
val_dataset = TensorDataset(
torch.FloatTensor(X_val),
torch.LongTensor(yh_val),
torch.LongTensor(yc_val)
)
train_loader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True)
val_loader = DataLoader(val_dataset, batch_size=batch_size)
# Initialize model
num_cancer_types = len(np.unique(y_cancer))
model = DualCNN(X.shape[1], num_cancer_types).to(device)
# Train model
train_model(model, train_loader, val_loader, device)
print("Training completed successfully!")
except Exception as e:
print(f"Error in main: {str(e)}")
print("Full error traceback:")
import traceback
traceback.print_exc()
if name == "main": main()
Log in or sign up for Devpost to join the conversation.