Projects
Master's Thesis: Transformer & RetNet
In ProgressResearching efficiency-vs-performance trade-offs between Transformer and RetNet architectures, with a focus on transfer learning to smaller models under limited compute. Broader research interests include multimodal models and agentic AI systems.
Tesla Stock Prediction
LSTM Time SeriesBuilt an LSTM neural network to forecast Tesla stock prices, with analysis of COVID-19's impact on model behavior. Full pipeline in Python: preprocessing, architecture design, training, and evaluation.
Wine Quality Prediction
Tabular MLPredicted wine quality from physicochemical features using tree-based models and XGBoost. Applied full EDA, feature importance analysis, and model comparison with scikit-learn and Matplotlib.
Attention vs. Retention, in one picture
A Transformer's attention lets every token look at every earlier token directly — exact recall, quadratic cost. RetNet's retention carries a running state instead — linear cost, but memory of distant tokens decays. Drag the slider, or click a token to move the highlighted query.
The gap explodes with scale: at 1,000 tokens, attention computes 499,500 pairwise links while retention makes 999 state updates. My thesis measures what that efficiency costs in accuracy — and when the trade is worth it.