IAM

ARTICLE

How I Prepared for DeepMind and Google AI Research Internship Interviews in 2019

In 2019, I interviewed for research internships at DeepMind and Google AI. I have been asked repeatedly about my preparation for and experience with these interviews. As internship applications at DeepMind have been opened recently, I thought it could be valuable to summarize my experience and recommendations in this article.

Introduction

Disclaimer: I interviewd for both DeepMind and Google AI internships at the end of 2019. Interview processes may have changed, especially due to the pandemic.

With the DeepMind internship program being open for applications, I thought it could be useful to summarize my experience and preparation for both DeepMind and Google AI interviews from late 2019. So in this article, I want to give a short overview of how I approached the interviews, focusing particularly on preparation and less so on the exact process and timing. While I only interviewed at DeepMind and Google AI, I expect this article to apply to many other research scientist internship (or even research engineering) programs, as well.

Specifically, I want to give a short overview of how I approached the whole interview process. Then, I want to describe how I prepared for the technical interviews — coding and research —, team interviews and culture/behavioral interviews.

Rough Process and my Approach

Generally speaking, interviews for research internships will have a coding component, a math/statistics/machine learning component, team interviews and potentially a culture/behavioral component. In the following, I want to give my perspective on these components and briefly summarize how I prepared for them.

Most important when talking about preparation is knowing your timeline. To some extent, you will have control over your timeline by moving interviews around or asking for more time for preparation. To a large part, however, the timeline will be dictated by the recruiting process at hand — that is, the company you are applying to. There are also strategic aspects to it. For example, during a PhD, you might want to avoid conference deadlines. At the same time, it might be reasonable to go through the process quicker depending on how internship positions are filled. These are aspects to keep in mind. In this article, however, I will mainly concentrate on preparation and not talk about the interview schedule.

My preparation process can roughly be broken down into an initial information gathering phase and an actual preparation phase. Information gathering is mostly about understanding the process (with the help of your recruiter and network), determining what (or who) to study and in what order/how much time to allocate for it. This can happen in parallel to drafting the application, or between submitting the application and getting feedback (although this might be quite short). In my case, this probably took 2-3 weeks besides my PhD work. Part of this also continued throughout the interviews if you make it beyond the technical interviews.

The actual preparation phase is studying and practicing everything that you identified as relevant for your interviews. This involves practicing coding interviews, brushing up your coding skills, studying basic math, statistics or machine learning and potentially reading up on specific topics you identified as important. How much time to spend on this is determined by your experience with interviews and the time available to you. Generally, you should aim to be confident in your skills and feel comfortable going into the interviews. If this is the first time going through these types of interviews, you should consider mock interviews to recognize when you're ready. In this case it might be worth allocating more time. In my case, I allocated a bit more than 2 weeks, aiming for 2-3h a day. As I also did some preparation on weekends, this might boil down to 3 weeks if you only work on workdays. Note that I went through coding interviews before and I prioritized this preparation over my PhD work for these weeks.

Technical Interviews

The biggest portion of preparation will go into the technical interviews, coding and math/statistics/machine learning:

Regarding coding, I found it useful to follow general resources for coding interviews aimed at software engineering roles. I followed the Cracking the Coding Interview book for most part and complemented it with topics that I felt are relevant — for example, tensor operations and libraries as commonly used in machine learning. In addition, I revisited basic software engineering principles such as code reviewing, testing, design patterns etc. However, there are many other books and resources you can find through a simple Google search. During the information gathering phase, you should gather and decide on the resources you want to use and I found it helpful to make a list of topics to practice. It also involves deciding on the programming language to focus on. Part of the list can be found below, or check out this Jupyter notebook to get an idea of what I revisited for my Python interviews. Note that I previously applied for software engineering internships, so this process was not new to me. See my previous article for details on this part.

As most of these resources will be targeted to software engineers, an important decision is how to trade-off the coding part and the research part — mathematics, statistics, and machine learning. This is particularly important when preparing under time constraints. While I believe coding to be an important part of the internship, I recommend practicing coding at least enough to meet the general "software engineer internship bar" but spend extra time and effort on the research part. This is for research scientist internships, of course. This is also something you can directly check with your recruiter. Some companies might have code comprehension interviews instead of full-fledged coding interviews for research internships, while others still resort to standard coding interviews (often conducted by employees outside the research organization).

Research interviews might be less standardized in the industry compared to coding interviews. Your recruiter should give you some indication of how to prepare. Generally, however, it will involve basic mathematics, statistics and machine learning/AI. In the information gathering phase, you should decide what you need to brush up on. While you might be comfortable working in a very specialized research area, I found these interviews are often about a profound understanding of the basics. I went through my old university lecture notes on calculus, statistics, linear algebra, numerical analysis, logic and complexity among others. Part of the list I came up with can be found below.

How exactly to do this will be very individual. Here is my process: I went through the slides and lecture notes I had from my bachelor and master studies. The main reason for this is because these were familiar in terms of structure and notation and I thought this is the quickest way to get back into the details. I went through my notes and essentially extracted (sometimes I literally screenshotted) the most relevant parts — important definitions and theorems, for example. For machine learning, as example, I used the summary slides from my first lecture (see here for a later version) and used Christopher Bishop's book which I used throughout my studies and had notes on.

Communication: An important aspect of both coding and research interviews is communication. I think that properly communicating a potential solution to a coding question is more important that having the most efficient and elegant solution. I recommend to practice coding interviews while explaining your thought process (to somebody — or nobody). This also manifests itself in the code you write. Take a couple of minutes to go over it in the end to clarify variable names, maybe add comments or check syntax. Ask questions. If you feel more comfortable thinking in silence, communicate this. The same applies, more or less, to research interviews, as well.

My list of topics to study:

  • Technical interviews:
    • Python:
      • data types
      • memory management & garbage collection
      • functions (by value and by reference, default values, kwargs)
      • local and global functions
      • return values
      • global and local variables
      • lambda expressions
      • argument lists
      • for, foreach, while, if else, inline if else
      • generators and yield
      • class definitions
      • private and public attributes/functions
      • static variables
      • self object
      • inheritance (overwriting)
      • constructors, destructors (definition, order)
      • magic functions
      • classes in classes
      • importing mechanisms
      • error handling
      • built-in data structures
      • built-in functions
    • Discrete (Hanke script):
      • Permutations
      • Combinatorics
      • Graphs
      • Connected components
    • Calculus (AFI Script):
      • Continuity (definition, theorems)
      • Differentiation (definition, core functions, various rules)
      • Integration (definition, rules, core functions)
      • Ordinary differential euqations
      • Multivariate functions
      • Taylor expansion
      • Norms, triangle inequality
      • Convolution
      • Variational methods
      • Maximum and minimum value problems
      • Trigonometric functions
      • Exponential function
    • Linear algebra (Hanke script):
      • Vector spaces
      • Matrices (trace, orthogonal, diagonal)
      • Linear, affine , projective transformations
      • Eigenvalues
      • Orthogonal group
      • Linear equation systems
    • Numerical analysis & optimization (Dahmen book):
      • Stability
      • LR deocmposition
      • QR decomposition
      • SVD decomposition
      • Eigenvalue decomposition
      • Least squares
      • Gradient descent (convergence, convex and non-convex)
      • Newton-method
    • Statistics:
      • Axioms of probability
      • Random variables
      • Discrete distributions
      • Expected value
      • Variance
      • Moments
      • Unbiased estimators (Bernoulli, Binomial)
      • Continuous distirbutions (Gaussian, Exponential, Dirac, Poisson, t-Student)
      • Maximum likelihood
      • hypothesis testing and confidence intervals
    • Logic (script):
      • Propositional logic
      • First-order logic
    • Algorithms & data structures (slides + Cormen book):
      • Arrays and linked lists, stacks, queues
      • Hash tables
      • Sorting (mergesort, quicksort, lower bound, radix sort)
      • Priority queues
      • Graphs (adjacency lists, arrays, matrices)
      • Breadth-first
      • Depths-first
      • DAGs
      • Distraer
      • Kruskal
      • Linear programming
      • Dynamic programming
      • Sink-source problems/maximum flow
      • Big-O (definition, theorems)
      • Worst-case analysis
      • Average-case analysis
    • Complexity (?):
      • Computability (definition)
      • Halting problem
      • Complexity class (definition, theorems)
      • P
      • NP
      • SPACE
      • NSPACE
      • Relationships
    • Software Engineering (slides SWE + project management):
      • Software processes (waterfall model)
      • Agile development and tools (Scrum, Kanban, pair programming, extreme programming)
      • Requirements
      • System design
      • Testing (coverage, unit testing, test first)
      • Patterns (?)
      • Project management
    • Machine learning (Bishop book + Sutton book + Leibe slides):
      • Curse of dimensionality (distances, norms, probabilities in high dimension)
      • Decision theory
      • Naive Bayes
      • Nearest neighbor
      • Kernel density estimation
      • Regularized least squares
      • Bayesian linear regression
      • Least squares classification
      • Fisher's discriminant
      • Logistic regression
      • Neural networks
        • Backpropagation
        • Universal approximators
      • SVMs
      • Gaussian processes
      • Graphical models (conditional independence, MRFs, sum-product and max-sum)
      • K-means
      • DBSCAN
      • Mean shift
      • PCA
      • PPCA
      • Monte Carlo
      • Markov Chain Monte Carlo
      • Gibbs Sampling
      • Boosting
      • Decision trees
      • Random forests
      • PAC learning
      • Reinforcement learning (?)
Additional topics to brush up on:
  • Interpretability (review paper and tutorial)
  • Fairness (tutorial paper)
  • Backdooring (regular papers), membership inference (review or tutorial), privacy (differentiable privacy, federated learning tutorial or review)
  • Transformers (tutorial)
  • 3D Vision (3D Geometry, Nerf, etc.)
  • Graph Neural Networks
  • ... ?

Team Matching

Following the technical interviews there will be some sort of interviews to find a fitting team. This will be varying a lot depending on the company, the organization within the company, or even your recruiter and your timing. This might also be least transparent. Thus, I want to focus mostly on the interviews themselves rather than how to navigate this phase. For research internships, I experienced two types of interviews: interviews with potential hosts and interviews with team members.

Potential hosts mainly want to find out whether they want to work with you. At the same time, you should decide whether you want to work with the host. This is probably the least well-defined interview. I had very open-ended and informal meetings as well as more interview-style meetings. In the end, however, both you and your host should have an idea of whether you want to work together. Therefore, I recommend a mixed strategy. First, you should be able to briefly summarize your background and highlight key research work that might be relevant for the specific host — more on that below. And second, think about questions to ask that help you decide on working with the host or not. However, keep in mind that this is a two-way street. As much as you want to decide whether you want to work with a host, he or she might have similar questions.

Examples: For the host, I usually asked several questions to get an idea of how the intership might look in practice. What are potential topics? Is the topic fixed or will you have some freedom to influence it or contribute your own ideas? Will your internship be part of a bigger project? Will it be a project mostly driven by you, or will you have to work with other researchers and engineers? Will the topic require working across teams? How does your host usually go about advising — will there be regular meetings, help with onboarding, etc.?

Then, there will be interviews with team members. These might be potential collaborators on the project or might be researchers and engineers from the same team but different projects. Again, you want to be able to highlight some of your research work and succinctly describe your background — essentially give a pitch of your CV, followed by a pitch of selected papers. The goal in these meetings should be to have a research discussion, ideally mixing some of your work with the interviewers work. For this it is important to get to know your interviewer's background. Check out recent publications, maybe you can find a Google Scholar profile, a CV or even a webpage, old talks, etc. Read some of the papers that you feel are close to yours. These interviewers will give your host an opinion on whether they can imaging working with you and, personally, I fared well with keeping these interviews casual, but focused on research that is interesting to both of you.

Examples: Here you can find my "pitch" from back in 2019. The pitch stayed the same, but for different interviews, I selected different papers from me to highlight and potentially discuss.

Culture and Behavioral Stuff

Sometimes there might also be an interview with a program manager, team coordinator or similar. In some books, these are referred to as culture or behavioral interviews. In my experience, however, it is more like a casual discussion that is by construction (due to the interviewer's role) not research focused. For you, it is an opportunity to learn more about how work at the company is organized (team structure, project milestones, regular meetings, office culture, etc.). For the interviewer, the goal is likely to learn how you work and whether that fits in the team. For preparation, I tried to learn as much as possible about the company and reflected on my own style of working.

Company culture might be difficult to grasp. If you know someone that you can talk to, this is the best way to learn about the company culture and what people value. Alternatively, you can find this out implicitly throughout the previous interviews. Finally, there is always their webpage and other public material. I tried to identify the main business purpose/mission of the company, read up on recent news and tried to find out how the company is viewed by the public. This is what I would refer to as the culture part of the interview.

Then, I prepared myself for questions about my personality and work style. When Googling, you will find many "standard" questions that may seem absurd. But I actually encountered some of them. Generally, I found it useful to reflect on how I handled specific situations. For example, is there an example where I demonstrated good teamwork or leadership? Were there difficult situations that I resolved in a good way? What mistakes did I learn the most from? In my experience, it is not about answering these particular questions, but it is about learning how to communicate your soft skills and experience. And I found this was necessary in several interviews I had. This part is referred to as behavioral interviews in some books.

My list of things/questions to consider:

  • Company mission and culture:
    • Identify the company's history and mission.
    • What are the key breakthroughs of the company?
    • How do these reflect the company's culture and mission.
    • How can I identify with the mission and goal of the company?
    • Read up on topics important for the company
    • Read up on recent news about the company
    • How is the company displayed in the public?
    • Can I learn something about company structure and organization?
    • ...
  • Behavioral: how situations are handled (to reveal personality and abilities):
    • What gets you excited?
    • When do I do my best work?
    • Plans for next years
    • General: use specific examples from experience so far;
    • STAR: Siutation - Task - Action - Result;
    • When did you solve logic to solve a problem?
    • How do you handle a challenge?
    • Did you ever make a risky decision?
    • Have you dealt with a company policy that you did not agree with?
    • Have you gone above the call of duty?
    • Have you convinced a team to work on a new project?
    • Have you handled difficult situations with a co-worker?
    • How do you handle disagreement?
    • Examples of teamwork
    • Stressful situation and working under pressure, meeting deadlines
    • Unpopular decision and how implemented
    • If you schedule gets interrupted?
    • Situation were I didn't have enough work to do?
    • Mistakes
    • Weaknesses
    • Strengths

Conclusion

Overall, I hope this article helps some future research interns with their interview preparation. Even though I had gone through coding interviews before, I found the process quite intimidating. However, I found that gathering sufficient information about the process and planning the preparation process properly can help make the interviews more comfortable and less stressful.

What is your opinion on this article? Let me know your thoughts on Twitter @davidstutz92 or LinkedIn in/davidstutz92.