Home
/
Blog
/
Developer Insights
/
Inside The Mind Of A Data Scientist

Inside The Mind Of A Data Scientist

Author
Kumari Trishya
Calendar Icon
January 22, 2021
Timer Icon
3 min read
Share

Problem:
There’s a port somewhere in the world that wants to maximize profits.

Approach:
Said port hires a data scientist to look at the numerous variables affecting ship movement and operational efficiency – factors that affect profitability in the long run.

The data scientist looks at how many ships enter the port on a daily basis, where they are loaded and unloaded, the size of ships coming in versus the length of the docks where they are anchored, the time lost when a ship of the wrong size enters a dock and then has to re-dock correctly, the number of port employees required to unload a single ship by length and type of cargo, the future plans for the port and the predicted volume of ships entering.

Then they begin their analysis.

Conclusion:
Our data hero announces that the port will have to hire at a rate of 3% every year to keep up with increasing volume. They also help the authorities set up a system that helps ships navigate to the correct dock and alerts authorities in advance when a ship is approaching. This leads to increased efficiency overall, better communication between the docks and the ships; thus decreasing time lost in re-docking, and increases profits for the port.

Accounting for seasonal variations in traffic, and the time and effort needed to train the staff in using the new navigation system, the data scientist predicts that the port can look at a probable profit increase of 20% in 3 years.

**The key word here is ‘probable’.**

Let’s read that first part again. The solution seems so simple, right? That simple solution, however, requires months of data crunching and historical analysis to create operational models for the future.

The end result in this scenario is a probability and not a number written in stone, because several factors (trade wars, a pandemic, oil prices, consumer demand) can affect the port’s operations. These are factors one cannot guarantee, or foresee, but a good data scientist is expected to account for all of these and still come up with a reliable prediction.

This is why good data scientists are so in-demand across the tech sector. Also, why assessing and hiring good data scientists is so hard.

Data scientists are not the same as generalist programmers

Assessing a data scientist is not the same as assessing another developer. The above example would have helped you understand the difference between the problems that a data scientist works on and those that a programmer solves.

There are differences even in the skill sets required for a data scientist role, and those required by other developers as illustrated below:

Data-Scientist-Assessments-Jupyter-Notebooks-HackerEarth

Traditional IDEs, therefore, don’t cut it for data scientists

Most IDEs include a source code editor, debugger, and compiler. They work perfectly for tech assessments for programmers and developers. Not for data science and machine learning assignments though.

In many data science problems, the solution can be a simple prediction or a ‘Yes/No’ answer. Or, if we go back to the question we started this blog with, it can be a prediction about the probability of achieving the desired goal. Is it going to rain in Atlanta tomorrow? Yes. Will my company grow 5X in the next two years? Ummm, there’s a 20% chance of doing that given you do these 10 other things well.

As we have already established, arriving at this answer requires hours of logical analysis. When assessing a data scientist for a job, therefore, recruiters and hiring managers need to be able to understand the logical choices the candidate made while arriving at the seemingly simple conclusion. A traditional IDE is not enough here.

SUBSCRIBE to the HackerEarth blog and enrich your monthly reading with our free e-newsletter – Fresh, insightful and awesome articles like these straight into your inbox from around the tech recruiting world!

Hence, Jupyter Notebooks

At HackerEarth, we have seen an increasing demand for Data Science and Machine Learning – a trend reflected in our year-end recruiter survey as well. To make data science assessments easier for recruiters, we have now integrated Jupyter Notebooks on our assessment platform, which helps recruiters get right inside the mind of the candidate they are trying to hire.

The Jupyter Notebook is an open-source web application that allows users to create and share documents containing live code, equations, visualizations, and narrative text. The easy-to-use, interactive data science environment provided by Jupyter works across several programming languages such as Python and R. Jupyter Notebooks not only work like an IDE, but also as a presentation or education tool, and are great for data science assessments where the candidate is required to answer questions in a visual format.

Here are some of the ways Jupyter Notebooks score over traditional IDEs:
  • Individual cells for better analysis

Jupyter Notebooks allow candidates to code using separate units or ‘cells’ that can be used independently of each other while writing code (denoted by red arrows in the image below). This makes it easier for candidates to compute how various data parameters work with each other and to add notes, or to partially write and test code.

This is essential for recruiters to understand the analytical approach taken by the candidate when solving a problem.

 jupyter-notebooks-hackerearth-assessments-cells.
  • Interactive elements for better data visualization

The Notebook offers an interactive shell with embeddable graphics and tables, reusable cells, and some other presentation features which are relevant to the job at hand. This enables candidates to present their output in a graphical format if needed; something that a traditional IDE does not support.

jupyter-notebooks-hackerearth-assessments-graphs.
  • Enhanced candidate experience

It is well known that candidates perform better when they are using a test environment they are familiar with. Notebooks are a preferred tool in the data science world. Using the Jupyter platform for an assessment ensures that your candidate is comfortable and ready, and is approaching a problem the way they would in real life.

jupyter notebooks - hackerearth assessments - benefits

Better data science assessments are made of these

When the candidate starts the assignment, they are given a choice to use the Monaco editor (IDE) or Jupyter Notebooks. The Notebooks use a dedicated machine to provide enough resources to each user. Thus by ensuring a dedicated machine for every assignment our candidates take, we affirm that the candidate has no restrictions and completely feels at home. This directly translates to better candidate output in the test, and an objective skill-based assessment process.

The most interesting bit about the Jupyter Notebook integration is the output section, which not only captures the final submission in CSV format but allows recruiters to review each and every step taken by the candidate as they solved the data problem before them.

So, even if a candidate gets a Yes/No prediction wrong, you can still review their work to see how they analyzed the data – the most crucial part of a data scientist’s role.

jupyter-notebooks-hackerearth-assessments-candidate-submission

Find better candidates with Jupyter Notebooks. Thank us later!

While data science as a field dates back to 1962 when mathematician John W. Tukey predicted the effect of modern-day electronic computing on data analysis as an empirical science. However, it reached the modern-day tech hiring lexicon only in recent years.

The trends we have seen tell us that tech jobs in AI (Artificial Intelligence), ML (Machine Learning), and Data Science would be the most in-demand roles in the future. With growing opportunities for AI and ML specialists in industries as diverse as banking, fintech, public safety, and healthcare, there will be a surge in these roles in the coming days. Today, every business big or small needs BIG DATA, and with the advent of various technologies that allow easy application of data science, all businesses are looking at using data to make their solutions smarter, their operations more efficient, and their user experiences more personalized.

This predicted surge in hiring also underlines the need to objectively assess and hire the best data scientists in the market. Traditional modes of evaluation do not do justice to the skills and expectations associated with this role. With the Jupyter notebook support on our HackerEarth Assessments platform, however, you can now assess and hire the best data scientists out there, and improve your business pipeline.

Try it out and let us know? You can even mail our product manager Akash Bhat (akash@hackerearth.com) to know more about this feature.

Recommended Read: HackerEarth’s Complete Guide to Hiring A Data Scientist

Subscribe to The HackerEarth Blog

Get expert tips, hacks, and how-tos from the world of tech recruiting to stay on top of your hiring!

Author
Kumari Trishya
Calendar Icon
January 22, 2021
Timer Icon
3 min read
Share

Hire top tech talent with our recruitment platform

Access Free Demo
Related reads

Discover more articles

Gain insights to optimize your developer recruitment process.

The Mobile Dev Hiring Landscape Just Changed

Revolutionizing Mobile Talent Hiring: The HackerEarth Advantage

The demand for mobile applications is exploding, but finding and verifying developers with proven, real-world skills is more difficult than ever. Traditional assessment methods often fall short, failing to replicate the complexities of modern mobile development.

Introducing a New Era in Mobile Assessment

At HackerEarth, we're closing this critical gap with two groundbreaking features, seamlessly integrated into our Full Stack IDE:

Article content

Now, assess mobile developers in their true native environment. Our enhanced Full Stack questions now offer full support for both Java and Kotlin, the core languages powering the Android ecosystem. This allows you to evaluate candidates on authentic, real-world app development skills, moving beyond theoretical knowledge to practical application.

Article content

Say goodbye to setup drama and tool-switching. Candidates can now build, test, and debug Android and React Native applications directly within the browser-based IDE. This seamless, in-browser experience provides a true-to-life evaluation, saving valuable time for both candidates and your hiring team.

Assess the Skills That Truly Matter

With native Android support, your assessments can now delve into a candidate's ability to write clean, efficient, and functional code in the languages professional developers use daily. Kotlin's rapid adoption makes proficiency in it a key indicator of a forward-thinking candidate ready for modern mobile development.

Breakup of Mobile development skills ~95% of mobile app dev happens through Java and Kotlin
This chart illustrates the importance of assessing proficiency in both modern (Kotlin) and established (Java) codebases.

Streamlining Your Assessment Workflow

The integrated mobile emulator fundamentally transforms the assessment process. By eliminating the friction of fragmented toolchains and complex local setups, we enable a faster, more effective evaluation and a superior candidate experience.

Old Fragmented Way vs. The New, Integrated Way
Visualize the stark difference: Our streamlined workflow removes technical hurdles, allowing candidates to focus purely on demonstrating their coding and problem-solving abilities.

Quantifiable Impact on Hiring Success

A seamless and authentic assessment environment isn't just a convenience, it's a powerful catalyst for efficiency and better hiring outcomes. By removing technical barriers, candidates can focus entirely on demonstrating their skills, leading to faster submissions and higher-quality signals for your recruiters and hiring managers.

A Better Experience for Everyone

Our new features are meticulously designed to benefit the entire hiring ecosystem:

For Recruiters & Hiring Managers:

  • Accurately assess real-world development skills.
  • Gain deeper insights into candidate proficiency.
  • Hire with greater confidence and speed.
  • Reduce candidate drop-off from technical friction.

For Candidates:

  • Enjoy a seamless, efficient assessment experience.
  • No need to switch between different tools or manage complex setups.
  • Focus purely on showcasing skills, not environment configurations.
  • Work in a powerful, professional-grade IDE.

Unlock a New Era of Mobile Talent Assessment

Stop guessing and start hiring the best mobile developers with confidence. Explore how HackerEarth can transform your tech recruiting.

Vibe Coding: Shaping the Future of Software

A New Era of Code

Vibe coding is a new method of using natural language prompts and AI tools to generate code. I have seen firsthand that this change makes software more accessible to everyone. In the past, being able to produce functional code was a strong advantage for developers. Today, when code is produced quickly through AI, the true value lies in designing, refining, and optimizing systems. Our role now goes beyond writing code; we must also ensure that our systems remain efficient and reliable.

From Machine Language to Natural Language

I recall the early days when every line of code was written manually. We progressed from machine language to high-level programming, and now we are beginning to interact with our tools using natural language. This development does not only increase speed but also changes how we approach problem solving. Product managers can now create working demos in hours instead of weeks, and founders have a clearer way of pitching their ideas with functional prototypes. It is important for us to rethink our role as developers and focus on architecture and system design rather than simply on typing c

The Promise and the Pitfalls

I have experienced both sides of vibe coding. In cases where the goal was to build a quick prototype or a simple internal tool, AI-generated code provided impressive results. Teams have been able to test new ideas and validate concepts much faster. However, when it comes to more complex systems that require careful planning and attention to detail, the output from AI can be problematic. I have seen situations where AI produces large volumes of code that become difficult to manage without significant human intervention.

AI-powered coding tools like GitHub Copilot and AWS’s Q Developer have demonstrated significant productivity gains. For instance, at the National Australia Bank, it’s reported that half of the production code is generated by Q Developer, allowing developers to focus on higher-level problem-solving . Similarly, platforms like Lovable or Hostinger Horizons enable non-coders to build viable tech businesses using natural language prompts, contributing to a shift where AI-generated code reduces the need for large engineering teams. However, there are challenges. AI-generated code can sometimes be verbose or lack the architectural discipline required for complex systems. While AI can rapidly produce prototypes or simple utilities, building large-scale systems still necessitates experienced engineers to refine and optimize the code.​

The Economic Impact

The democratization of code generation is altering the economic landscape of software development. As AI tools become more prevalent, the value of average coding skills may diminish, potentially affecting salaries for entry-level positions. Conversely, developers who excel in system design, architecture, and optimization are likely to see increased demand and compensation.​
Seizing the Opportunity

Vibe coding is most beneficial in areas such as rapid prototyping and building simple applications or internal tools. It frees up valuable time that we can then invest in higher-level tasks such as system architecture, security, and user experience. When used in the right context, AI becomes a helpful partner that accelerates the development process without replacing the need for skilled engineers.

This is revolutionizing our craft, much like the shift from machine language to assembly to high-level languages did in the past. AI can churn out code at lightning speed, but remember, “Any fool can write code that a computer can understand. Good programmers write code that humans can understand.” Use AI for rapid prototyping, but it’s your expertise that transforms raw output into robust, scalable software. By honing our skills in design and architecture, we ensure our work remains impactful and enduring. Let’s continue to learn, adapt, and build software that stands the test of time.​

Ready to streamline your recruitment process? Get a free demo to explore cutting-edge solutions and resources for your hiring needs.

Guide to Conducting Successful System Design Interviews in 2025

What is Systems Design?

Systems Design is an all encompassing term which encapsulates both frontend and backend components harmonized to define the overall architecture of a product.

Designing robust and scalable systems requires a deep understanding of application, architecture and their underlying components like networks, data, interfaces and modules.

Systems Design, in its essence, is a blueprint of how software and applications should work to meet specific goals. The multi-dimensional nature of this discipline makes it open-ended – as there is no single one-size-fits-all solution to a system design problem.

What is a System Design Interview?

Conducting a System Design interview requires recruiters to take an unconventional approach and look beyond right or wrong answers. Recruiters should aim for evaluating a candidate’s ‘systemic thinking’ skills across three key aspects:

How they navigate technical complexity and navigate uncertainty
How they meet expectations of scale, security and speed
How they focus on the bigger picture without losing sight of details

This assessment of the end-to-end thought process and a holistic approach to problem-solving is what the interview should focus on.

What are some common topics for a System Design Interview

System design interview questions are free-form and exploratory in nature where there is no right or best answer to a specific problem statement. Here are some common questions:

How would you approach the design of a social media app or video app?

What are some ways to design a search engine or a ticketing system?

How would you design an API for a payment gateway?

What are some trade-offs and constraints you will consider while designing systems?

What is your rationale for taking a particular approach to problem solving?

Usually, interviewers base the questions depending on the organization, its goals, key competitors and a candidate’s experience level.

For senior roles, the questions tend to focus on assessing the computational thinking, decision making and reasoning ability of a candidate. For entry level job interviews, the questions are designed to test the hard skills required for building a system architecture.

The Difference between a System Design Interview and a Coding Interview

If a coding interview is like a map that takes you from point A to Z – a systems design interview is like a compass which gives you a sense of the right direction.

Here are three key difference between the two:

Coding challenges follow a linear interviewing experience i.e. candidates are given a problem and interaction with recruiters is limited. System design interviews are more lateral and conversational, requiring active participation from interviewers.

Coding interviews or challenges focus on evaluating the technical acumen of a candidate whereas systems design interviews are oriented to assess problem solving and interpersonal skills.

Coding interviews are based on a right/wrong approach with ideal answers to problem statements while a systems design interview focuses on assessing the thought process and the ability to reason from first principles.

How to Conduct an Effective System Design Interview

One common mistake recruiters make is that they approach a system design interview with the expectations and preparation of a typical coding interview.
Here is a four step framework technical recruiters can follow to ensure a seamless and productive interview experience:

Step 1: Understand the subject at hand

  • Develop an understanding of basics of system design and architecture
  • Familiarize yourself with commonly asked systems design interview questions
  • Read about system design case studies for popular applications
  • Structure the questions and problems by increasing magnitude of difficulty

Step 2: Prepare for the interview

  • Plan the extent of the topics and scope of discussion in advance
  • Clearly define the evaluation criteria and communicate expectations
  • Quantify constraints, inputs, boundaries and assumptions
  • Establish the broader context and a detailed scope of the exercise

Step 3: Stay actively involved

  • Ask follow-up questions to challenge a solution
  • Probe candidates to gauge real-time logical reasoning skills
  • Make it a conversation and take notes of important pointers and outcomes
  • Guide candidates with hints and suggestions to steer them in the right direction

Step 4: Be a collaborator

  • Encourage candidates to explore and consider alternative solutions
  • Work with the candidate to drill the problem into smaller tasks
  • Provide context and supporting details to help candidates stay on track
  • Ask follow-up questions to learn about the candidate’s experience

Technical recruiters and hiring managers should aim for providing an environment of positive reinforcement, actionable feedback and encouragement to candidates.

Evaluation Rubric for Candidates

Facilitate Successful System Design Interview Experiences with FaceCode

FaceCode, HackerEarth’s intuitive and secure platform, empowers recruiters to conduct system design interviews in a live coding environment with HD video chat.

FaceCode comes with an interactive diagram board which makes it easier for interviewers to assess the design thinking skills and conduct communication assessments using a built-in library of diagram based questions.

With FaceCode, you can combine your feedback points with AI-powered insights to generate accurate, data-driven assessment reports in a breeze. Plus, you can access interview recordings and transcripts anytime to recall and trace back the interview experience.

Learn how FaceCode can help you conduct system design interviews and boost your hiring efficiency.

Top Products

Explore HackerEarth’s top products for Hiring & Innovation

Discover powerful tools designed to streamline hiring, assess talent efficiently, and run seamless hackathons. Explore HackerEarth’s top products that help businesses innovate and grow.
Frame
Hackathons
Engage global developers through innovation
Arrow
Frame 2
Assessments
AI-driven advanced coding assessments
Arrow
Frame 3
FaceCode
Real-time code editor for effective coding interviews
Arrow
Frame 4
L & D
Tailored learning paths for continuous assessments
Arrow
Get A Free Demo