Martin Marek

I am a PhD student in Machine Learning at NYU, working with Andrew Gordon Wilson and Pavel Izmailov. My research focuses on empirical contributions to the science of deep learning.

I'm currently interning at Together AI. Previously I studied Statistics and Machine Learning at UCL and interned as a Quant Researcher at QRT and ML Engineer at Snapchat.

Links

GitHub, Google Scholar, Twitter

Email me: martin.m@nyu.edu

Selected papers

Forgetting in Language Models:
Capacity, Optimization, and Self-Generated Replay
M. Marek, D. Cho, S. Qiu, R. Chunara, P. Izmailov, A. G. Wilson
[arXiv, PDF, GitHub, Twitter]
Small Batch Size Training for Language Models:
When Vanilla SGD Works, and Why Gradient Accumulation Is Wasteful
M. Marek, S. Lotfi, A. Somasundaram, A. G. Wilson, M. Goldblum
[arXiv, PDF, GitHub, Twitter, Blog]
Can a Confident Prior Replace a Cold Posterior?
M. Marek, B. Paige, P. Izmailov
[arXiv, PDF, GitHub, Twitter]

Personal projects

Burst Photo
Mac app that brings “night mode” to any camera.
Implements HDR+ from scratch in Metal Shading Language.
[Website, GitHub, DPReview, PetaPixel, SonyAlphaRumors]

Website design inspired by Pavel Izmailov.