When should I use deep learning instead of classical machine learning?
Use deep learning for unstructured data, such as images, text and audio, where it learns useful features directly from raw inputs. For tabular data, classical methods are often better: Grinsztajn, Oyallon and Varoquaux (2022) found tree-based models still outperformed deep learning on medium-sized tabular datasets. Choose by your data type and dataset size, not by which method sounds more advanced.
What is transfer learning, and why should I use it?
Transfer learning starts from a model already trained on a large dataset, such as ResNet for images (He et al., 2016) or BERT for text (Devlin et al., 2019), then fine-tunes it on your smaller dataset. It usually needs far less data and compute than training from scratch, and often performs better, which makes it the realistic choice for most dissertations.
How do I prevent overfitting on a small dataset?
Start from a pretrained model, and use data augmentation to create realistic variations of your training examples. Add regularisation, such as dropout (Srivastava et al., 2014) and weight decay, and use early stopping based on validation performance. Monitor training and validation curves: if training performance keeps improving while validation performance falls, the model is overfitting.
Should I use TensorFlow or PyTorch?
Both are mature and widely used, and either is fine for a dissertation. PyTorch is common in research code, so it helps when you build on a published method's code. TensorFlow with its Keras interface offers a simple, high-level way to build standard models. Choose the framework your supervisor uses or the papers you build on use.
How do I make deep learning results reproducible?
Set random seeds for every library you use, record library versions and hardware, and save your configuration and model checkpoints. Some GPU operations are not fully deterministic, so results can still vary slightly. Train each model several times and report the mean and standard deviation, not your single best run.
How do I compare deep learning models fairly?
Use the same data splits, preprocessing and evaluation metrics for every model, and give each a similar tuning effort and compute budget. Always include a simple baseline. Report results across several runs, and use statistical tests where differences are small; our computer science subject page explains which tests to use.