With perfect knowledge of the environment, reinforcement learning can be used to plan the behavior of an agent. In this post, I use gridworld to demonstrate three dynamic programming algorithms for Markov decision processes: policy evaluation, policy iteration, and value iteration.
Writing high-quality software can be hard. Luckily, there are principles that can guide object-oriented software design such as the SOLID principles, which consist of the single-responsibility principle, the open-closed principle, the Liskov substitution principle, the interface segregation principle, and the dependency inversion principle. In this post, I give a short explanation of each principle and give practical examples of their application.
Doing a PhD requires considerable scientific and technical skills. To be a successful PhD student, these hard skills must be complemented by an array of soft skills. In this post, I delineate what I find are the most important soft skills for PhD students.
Staticman makes it easy to implement commenting systems. However, spam can be a big problem. One way to prevent comments from bots is to integrate ReCAPTCHA and Staticman. In this post, I delineate my journey towards ReCAPTCHA verification.
Transitioning from academia to industry can be challenging. Based on working as a data scientist in research and as a DevOps engineer in industry, I share what I find are the greatest differences between working in academia vs industry. Finally, I offer some tips on how to prepare for an industry job.
Bioinformatics is an interdisciplinary field at the junction of computer science and biology. Considering aspects such as job options, however, is it worth studying bioinformatics?
The way in which academic software is developed differs starkly from the way that software is engineered in industry. In this article, I summarize the main differences between academic and professional software development and reveal how academics can up their game.
Forecasting is a powerful technique for time-series data. Here, I investigate the most common variants of forecasting algorithms: ARMA, ARIMA, SARIMA, and ARIMAX, which are primarily based on autocorrelation and moving averages.
Prediction and forecasting are similar, yet distinct areas for which machine learning techniques can be used. Here, I differentiate the two approaches using weather forecasting as an example.
ROC and precision-recall curves are a staple for the interpretation of binary classifiers. This post gives an intuition on how these curves are constructed and their associated AUCs are interpreted.