scientific problem solving pdf

If you're seeing this message, it means we're having trouble loading external resources on our website.

If you're behind a web filter, please make sure that the domains *.kastatic.org and *.kasandbox.org are unblocked.

To log in and use all the features of Khan Academy, please enable JavaScript in your browser.

Biology library

Course: biology library   >   unit 1, the scientific method.

• Controlled experiments
• The scientific method and experimental design

Introduction

• Make an observation.
• Ask a question.
• Form a hypothesis , or testable explanation.
• Make a prediction based on the hypothesis.
• Test the prediction.
• Iterate: use the results to make new hypotheses or predictions.

Scientific method example: Failure to toast

1. make an observation., 2. ask a question., 3. propose a hypothesis., 4. make predictions., 5. test the predictions..

• If the toaster does toast, then the hypothesis is supported—likely correct.
• If the toaster doesn't toast, then the hypothesis is not supported—likely wrong.

Logical possibility

Practical possibility, building a body of evidence, 6. iterate..

• If the hypothesis was supported, we might do additional tests to confirm it, or revise it to be more specific. For instance, we might investigate why the outlet is broken.
• If the hypothesis was not supported, we would come up with a new hypothesis. For instance, the next hypothesis might be that there's a broken wire in the toaster.

Want to join the conversation?

• Upvote Button navigates to signup page
• Downvote Button navigates to signup page
• Flag Button navigates to signup page

• school Campus Bookshelves
• menu_book Bookshelves
• perm_media Learning Objects
• login Login
• how_to_reg Request Instructor Account
• hub Instructor Commons

Margin Size

• Download Page (PDF)
• Download Full Book (PDF)
• Periodic Table
• Physics Constants
• Scientific Calculator
• Reference & Cite
• Tools expand_more
• Readability

selected template will load here

This action is not available.

1.12: Scientific Problem Solving

• Last updated
• Save as PDF
• Page ID 52325

\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)

\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)

\( \newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\)

( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\)

\( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)

\( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\)

\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)

\( \newcommand{\Span}{\mathrm{span}}\)

\( \newcommand{\id}{\mathrm{id}}\)

\( \newcommand{\kernel}{\mathrm{null}\,}\)

\( \newcommand{\range}{\mathrm{range}\,}\)

\( \newcommand{\RealPart}{\mathrm{Re}}\)

\( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)

\( \newcommand{\Argument}{\mathrm{Arg}}\)

\( \newcommand{\norm}[1]{\| #1 \|}\)

\( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\AA}{\unicode[.8,0]{x212B}}\)

\( \newcommand{\vectorA}[1]{\vec{#1}}      % arrow\)

\( \newcommand{\vectorAt}[1]{\vec{\text{#1}}}      % arrow\)

\( \newcommand{\vectorB}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)

\( \newcommand{\vectorC}[1]{\textbf{#1}} \)

\( \newcommand{\vectorD}[1]{\overrightarrow{#1}} \)

\( \newcommand{\vectorDt}[1]{\overrightarrow{\text{#1}}} \)

\( \newcommand{\vectE}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash{\mathbf {#1}}}} \)

How can we use problem solving in our everyday routines?

One day you wake up and realize your clock radio did not turn on to get you out of bed. You are puzzled, so you decide to find out what happened. You list three possible explanations:

• There was a power failure and your radio cannot turn on.
• Your little sister turned it off as a joke.
• You did not set the alarm last night.

Upon investigation, you find that the clock is on, so there is no power failure. Your little sister was spending the night with a friend and could not have turned the alarm off. You notice that the alarm is not set—your forgetfulness made you late. You have used the scientific method to answer a question.

Scientific Problem Solving

Humans have always wondered about the world around them. One of the questions of interest was (and still is): what is this world made of? Chemistry has been defined in various ways as the study of matter. What matter consists of has been a source of debate over the centuries. One of the key areas for this debate in the Western world was Greek philosophy.

The basic approach of the Greek philosophers was to discuss and debate the questions they had about the world. There was no gathering of information to speak of, just talking. As a result, several ideas about matter were put forth, but never resolved. The first philosopher to carry out the gathering of data was Aristotle (384-322 B.C.). He recorded many observations on the weather, on plant and animal life and behavior, on physical motions, and a number of other topics. Aristotle could probably be considered the first "real" scientist, because he made systematic observations of nature and tried to understand what he was seeing.

Inductive and Deductive Reasoning

Two approaches to logical thinking developed over the centuries. These two methods are inductive reasoning and deductive reasoning . Inductive reasoning involves getting a collection of specific examples and drawing a general conclusion from them. Deductive reasoning takes a general principle and then draws a specific conclusion from the general concept. Both are used in the development of scientific ideas.

Inductive reasoning first involves the collection of data: "If I add sodium metal to water, I observe a very violent reaction. Every time I repeat the process, I see the same thing happen." A general conclusion is drawn from these observations: the addition of sodium to water results in a violent reaction.

In deductive reasoning, a specific prediction is made based on a general principle. One general principle is that acids turn blue litmus paper red. Using the deductive reasoning process, one might predict: "If I have a bottle of liquid labeled 'acid', I expect the litmus paper to turn red when I immerse it in the liquid."

The Idea of the Experiment

Inductive reasoning is at the heart of what is now called the " scientific method ." In European culture, this approach was developed mainly by Francis Bacon (1561-1626), a British scholar. He advocated the use of inductive reasoning in every area of life, not just science. The scientific method, as developed by Bacon and others, involves several steps:

• Ask a question - identify the problem to be considered.
• Make observations - gather data that pertains to the question.
• Propose an explanation (a hypothesis) for the observations.
• Make new observations to test the hypothesis further.

Note that this should not be considered a "cookbook" for scientific research. Scientists do not sit down with their daily "to do" list and write down these steps. The steps may not necessarily be followed in order. But this does provide a general idea of how scientific research is usually done.

When a hypothesis is confirmed repeatedly, it eventually becomes a theory—a general principle that is offered to explain natural phenomena. Note a key word— explain , or  explanation . A theory offers a description of why something happens. A law, on the other hand, is a statement that is always true, but offers no explanation as to why. The law of gravity says a rock will fall when dropped, but does not explain why (gravitational theory is very complex and incomplete at present). The kinetic molecular theory of gases, on the other hand, states what happens when a gas is heated in a closed container (the pressure increases), but also explains why (the motions of the gas molecules are increased due to the change in temperature). Theories do not get "promoted" to laws, because laws do not answer the "why" question.

• The early Greek philosophers spent their time talking about nature, but did little or no actual exploration or investigation.
• Inductive reasoning - to develop a general conclusion from a collection of observations.
• Deductive reasoning - to make a specific statement based on a general principle.
• Scientific method - a process of observation, developing a hypothesis, and testing that hypothesis.
• What was the basic shortcoming of the Greek philosophers approach to studying the material world?
• How did Aristotle improve the approach?
• Define “inductive reasoning” and give an example.
• Define “deductive reasoning” and give an example.
• What is the difference between a hypothesis and a theory?
• What is the difference between a theory and a law?

When you choose to publish with PLOS, your research makes an impact. Make your work accessible to all, without restrictions, and accelerate scientific discovery with options like preprints and published peer review that make your work more Open.

• PLOS Biology
• PLOS Climate
• PLOS Complex Systems
• PLOS Computational Biology
• PLOS Digital Health
• PLOS Genetics
• PLOS Global Public Health
• PLOS Medicine
• PLOS Mental Health
• PLOS Neglected Tropical Diseases
• PLOS Pathogens
• PLOS Sustainability and Transformation
• PLOS Collections
• About This Blog
• Official PLOS Blog
• EveryONE Blog
• Speaking of Medicine
• PLOS Biologue
• Absolutely Maybe
• DNA Science
• PLOS ECR Community
• All Models Are Wrong
• About PLOS Blogs

A Guide to Using the Scientific Method in Everyday Life

The  scientific method —the process used by scientists to understand the natural world—has the merit of investigating natural phenomena in a rigorous manner. Working from hypotheses, scientists draw conclusions based on empirical data. These data are validated on large-scale numbers and take into consideration the intrinsic variability of the real world. For people unfamiliar with its intrinsic jargon and formalities, science may seem esoteric. And this is a huge problem: science invites criticism because it is not easily understood. So why is it important, then, that every person understand how science is done?

Because the scientific method is, first of all, a matter of logical reasoning and only afterwards, a procedure to be applied in a laboratory.

Individuals without training in logical reasoning are more easily victims of distorted perspectives about themselves and the world. An example is represented by the so-called “ cognitive biases ”—systematic mistakes that individuals make when they try to think rationally, and which lead to erroneous or inaccurate conclusions. People can easily  overestimate the relevance  of their own behaviors and choices. They can  lack the ability to self-estimate the quality of their performances and thoughts . Unconsciously, they could even end up selecting only the arguments  that support their hypothesis or beliefs . This is why the scientific framework should be conceived not only as a mechanism for understanding the natural world, but also as a framework for engaging in logical reasoning and discussion.

A brief history of the scientific method

The scientific method has its roots in the sixteenth and seventeenth centuries. Philosophers Francis Bacon and René Descartes are often credited with formalizing the scientific method because they contrasted the idea that research should be guided by metaphysical pre-conceived concepts of the nature of reality—a position that, at the time,  was highly supported by their colleagues . In essence, Bacon thought that  inductive reasoning based on empirical observation was critical to the formulation of hypotheses  and the  generation of new understanding : general or universal principles describing how nature works are derived only from observations of recurring phenomena and data recorded from them. The inductive method was used, for example, by the scientist Rudolf Virchow to formulate the third principle of the notorious  cell theory , according to which every cell derives from a pre-existing one. The rationale behind this conclusion is that because all observations of cell behavior show that cells are only derived from other cells, this assertion must be always true. 

Inductive reasoning, however, is not immune to mistakes and limitations. Referring back to cell theory, there may be rare occasions in which a cell does not arise from a pre-existing one, even though we haven’t observed it yet—our observations on cell behavior, although numerous, can still benefit from additional observations to either refute or support the conclusion that all cells arise from pre-existing ones. And this is where limited observations can lead to erroneous conclusions reasoned inductively. In another example, if one never has seen a swan that is not white, they might conclude that all swans are white, even when we know that black swans do exist, however rare they may be.  

The universally accepted scientific method, as it is used in science laboratories today, is grounded in  hypothetico-deductive reasoning . Research progresses via iterative empirical testing of formulated, testable hypotheses (formulated through inductive reasoning). A testable hypothesis is one that can be rejected (falsified) by empirical observations, a concept known as the  principle of falsification . Initially, ideas and conjectures are formulated. Experiments are then performed to test them. If the body of evidence fails to reject the hypothesis, the hypothesis stands. It stands however until and unless another (even singular) empirical observation falsifies it. However, just as with inductive reasoning, hypothetico-deductive reasoning is not immune to pitfalls—assumptions built into hypotheses can be shown to be false, thereby nullifying previously unrejected hypotheses. The bottom line is that science does not work to prove anything about the natural world. Instead, it builds hypotheses that explain the natural world and then attempts to find the hole in the reasoning (i.e., it works to disprove things about the natural world).

How do scientists test hypotheses?

Controlled experiments

The word “experiment” can be misleading because it implies a lack of control over the process. Therefore, it is important to understand that science uses controlled experiments in order to test hypotheses and contribute new knowledge. So what exactly is a controlled experiment, then? 

Let us take a practical example. Our starting hypothesis is the following: we have a novel drug that we think inhibits the division of cells, meaning that it prevents one cell from dividing into two cells (recall the description of cell theory above). To test this hypothesis, we could treat some cells with the drug on a plate that contains nutrients and fuel required for their survival and division (a standard cell biology assay). If the drug works as expected, the cells should stop dividing. This type of drug might be useful, for example, in treating cancers because slowing or stopping the division of cells would result in the slowing or stopping of tumor growth.

Although this experiment is relatively easy to do, the mere process of doing science means that several experimental variables (like temperature of the cells or drug, dosage, and so on) could play a major role in the experiment. This could result in a failed experiment when the drug actually does work, or it could give the appearance that the drug is working when it is not. Given that these variables cannot be eliminated, scientists always run control experiments in parallel to the real ones, so that the effects of these other variables can be determined.  Control experiments  are designed so that all variables, with the exception of the one under investigation, are kept constant. In simple terms, the conditions must be identical between the control and the actual experiment.     

Coming back to our example, when a drug is administered it is not pure. Often, it is dissolved in a solvent like water or oil. Therefore, the perfect control to the actual experiment would be to administer pure solvent (without the added drug) at the same time and with the same tools, where all other experimental variables (like temperature, as mentioned above) are the same between the two (Figure 1). Any difference in effect on cell division in the actual experiment here can be attributed to an effect of the drug because the effects of the solvent were controlled.

In order to provide evidence of the quality of a single, specific experiment, it needs to be performed multiple times in the same experimental conditions. We call these multiple experiments “replicates” of the experiment (Figure 2). The more replicates of the same experiment, the more confident the scientist can be about the conclusions of that experiment under the given conditions. However, multiple replicates under the same experimental conditions  are of no help  when scientists aim at acquiring more empirical evidence to support their hypothesis. Instead, they need  independent experiments  (Figure 3), in their own lab and in other labs across the world, to validate their results. 

Often times, especially when a given experiment has been repeated and its outcome is not fully clear, it is better  to find alternative experimental assays  to test the hypothesis. 

Applying the scientific approach to everyday life

So, what can we take from the scientific approach to apply to our everyday lives?

A few weeks ago, I had an agitated conversation with a bunch of friends concerning the following question: What is the definition of intelligence?

Defining “intelligence” is not easy. At the beginning of the conversation, everybody had a different, “personal” conception of intelligence in mind, which – tacitly – implied that the conversation could have taken several different directions. We realized rather soon that someone thought that an intelligent person is whoever is able to adapt faster to new situations; someone else thought that an intelligent person is whoever is able to deal with other people and empathize with them. Personally, I thought that an intelligent person is whoever displays high cognitive skills, especially in abstract reasoning. 

The scientific method has the merit of providing a reference system, with precise protocols and rules to follow. Remember: experiments must be reproducible, which means that an independent scientists in a different laboratory, when provided with the same equipment and protocols, should get comparable results.  Fruitful conversations as well need precise language, a kind of reference vocabulary everybody should agree upon, in order to discuss about the same “content”. This is something we often forget, something that was somehow missing at the opening of the aforementioned conversation: even among friends, we should always agree on premises, and define them in a rigorous manner, so that they are the same for everybody. When speaking about “intelligence”, we must all make sure we understand meaning and context of the vocabulary adopted in the debate (Figure 4, point 1).  This is the first step of “controlling” a conversation.

There is another downside that a discussion well-grounded in a scientific framework would avoid. The mistake is not structuring the debate so that all its elements, except for the one under investigation, are kept constant (Figure 4, point 2). This is particularly true when people aim at making comparisons between groups to support their claim. For example, they may try to define what intelligence is by comparing the  achievements in life of different individuals: “Stephen Hawking is a brilliant example of intelligence because of his great contribution to the physics of black holes”. This statement does not help to define what intelligence is, simply because it compares Stephen Hawking, a famous and exceptional physicist, to any other person, who statistically speaking, knows nothing about physics. Hawking first went to the University of Oxford, then he moved to the University of Cambridge. He was in contact with the most influential physicists on Earth. Other people were not. All of this, of course, does not disprove Hawking’s intelligence; but from a logical and methodological point of view, given the multitude of variables included in this comparison, it cannot prove it. Thus, the sentence “Stephen Hawking is a brilliant example of intelligence because of his great contribution to the physics of black holes” is not a valid argument to describe what intelligence is. If we really intend to approximate a definition of intelligence, Steven Hawking should be compared to other physicists, even better if they were Hawking’s classmates at the time of college, and colleagues afterwards during years of academic research. 

In simple terms, as scientists do in the lab, while debating we should try to compare groups of elements that display identical, or highly similar, features. As previously mentioned, all variables – except for the one under investigation – must be kept constant.

This insightful piece  presents a detailed analysis of how and why science can help to develop critical thinking.

In a nutshell

Here is how to approach a daily conversation in a rigorous, scientific manner:

• First discuss about the reference vocabulary, then discuss about the content of the discussion.  Think about a researcher who is writing down an experimental protocol that will be used by thousands of other scientists in varying continents. If the protocol is rigorously written, all scientists using it should get comparable experimental outcomes. In science this means reproducible knowledge, in daily life this means fruitful conversations in which individuals are on the same page. 
• Adopt “controlled” arguments to support your claims.  When making comparisons between groups, visualize two blank scenarios. As you start to add details to both of them, you have two options. If your aim is to hide a specific detail, the better is to design the two scenarios in a completely different manner—it is to increase the variables. But if your intention is to help the observer to isolate a specific detail, the better is to design identical scenarios, with the exception of the intended detail—it is therefore to keep most of the variables constant. This is precisely how scientists ideate adequate experiments to isolate new pieces of knowledge, and how individuals should orchestrate their thoughts in order to test them and facilitate their comprehension to others.   

Not only the scientific method should offer individuals an elitist way to investigate reality, but also an accessible tool to properly reason and discuss about it.

Edited by Jason Organ, PhD, Indiana University School of Medicine.

Simone is a molecular biologist on the verge of obtaining a doctoral title at the University of Ulm, Germany. He is Vice-Director at Culturico (https://culturico.com/), where his writings span from Literature to Sociology, from Philosophy to Science. His writings recently appeared in Psychology Today, openDemocracy, Splice Today, Merion West, Uncommon Ground and The Society Pages. Follow Simone on Twitter: @simredaelli

• Pingback: Case Studies in Ethical Thinking: Day 1 | Education & Erudition

This has to be the best article I have ever read on Scientific Thinking. I am presently writing a treatise on how Scientific thinking can be adopted to entreat all situations.And how, a 4 year old child can be taught to adopt Scientific thinking, so that, the child can look at situations that bothers her and she could try to think about that situation by formulating the right questions. She may not have the tools to find right answers? But, forming questions by using right technique ? May just make her find a way to put her mind to rest even at that level. That is why, 4 year olds are often “eerily: (!)intelligent, I have iften been intimidated and plain embarrassed to see an intelligent and well spoken 4 year old deal with celibrity ! Of course, there are a lot of variables that have to be kept in mind in order to train children in such controlled thinking environment, as the screenplay of little Sheldon shows. Thanking the author with all my heart – #ershadspeak #wearescience #weareallscientists Ershad Khandker

Simone, thank you for this article. I have the idea that I want to apply what I learned in Biology to everyday life. You addressed this issue, and have given some basic steps in using the scientific method.

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Save my name and email for the next time I comment.

By Ashley Moses, edited by Andrew S. Cale Each year, millions of scientific research papers are published. Virtually none of them can…

By Ana Santos-Carvalho and Carolina Lebre, edited by Andrew S. Cale Excessive use of technical jargon can be a significant barrier to…

By Ryan McRae and Briana Pobiner, edited by Andrew S. Cale In 2023, the field of human evolution benefited from a plethora…

Problem Solving

• Reference work entry
• pp 2680–2683
• Cite this reference work entry

• David H. Jonassen 2 &
• Woei Hung 3  

1890 Accesses

12 Citations

Cognition ; Problem typology ; Problem-based learning ; Problems ; Reasoning

Problem solving is the process of constructing and applying mental representations of problems to finding solutions to those problems that are encountered in nearly every context.

Theoretical Background

Problem solving is the process of articulating solutions to problems. Problems have two critical attributes. First, a problem is an unknown in some context. That is, there is a situation in which there is something that is unknown (the difference between a goal state and a current state). Those situations vary from algorithmic math problems to vexing and complex social problems, such as violence in society (see Problem Typology ). Second, finding or solving for the unknown must have some social, cultural, or intellectual value. That is, someone believes that it is worth finding the unknown. If no one perceives an unknown or a need to determine an unknown, there is no perceived problem. Finding...

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Durable hardcover edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Bransford, J., & Stein, B. S. (1984). The IDEAL problem solver: A guide for improving thinking, learning, and creativity . New York: WH Freeman.

Google Scholar  

Frensch, P. A., & Funke, J. (Eds.). (1995). Complex problem solving: The European perspective . Hillsdale: Erlbaum.

Gick, M. L., & Holyoak, K. J. (1983). Schema induction and analogical transfer. Cognitive Psychology, 15 , 1–38.

Article   Google Scholar  

Jonassen, D. H. (2010). Learning to solve problems: A handbook . New York: Routledge.

Jonassen, D. H., & Hung, W. (2008). All problems are not equal: Implications for PBL. Interdisciplinary Journal of Problem-Based Learning, 2 (2), 6–28.

Jonassen, D. H. (2000). Toward a design theory of problem solving. Educational Technology: Research & Development, 48 (4), 63–85.

Jonassen, D. H. (2011). Learning to solve problems: A handbook for designing problem-solving learning environments . New York: Routledge.

Klein, G. A. (1998). Sources of power: How people make decisions . Cambridge, MA: MIT Press.

Lehman, D., Lempert, R., & Nisbett, R. E. (1988). The effects of graduate training on reasoning: Formal discipline and thinking about everyday-life events. Educational Psychologist, 43 , 431–442.

Newell, A., & Simon, H. (1972). Human problem solving . Englewood Cliffs: Prentice Hall.

Rumelhart, D. E., & Norman, D. A. (1988). Representation in memory. In R. C. Atkinson, R. J. Herrnstein, G. Lindzey, & R. D. Luce (Eds.), Steven’s handbook of experimental psychology (Learning and cognition 2nd ed., Vol. 2, pp. 511–587). New York: Wiley.

Sinnott, J. D. (1989). Everyday problem solving: Theory and applications (pp. 72–99). New York: Praeger.

Wood, P. K. (1983). Inquiring systems and problem structures: Implications for cognitive development. Human Development, 26 , 249–265.

Download references

Author information

Authors and affiliations.

School of Information Science and Learning Technologies, University of Missouri, 221C Townsend Hall, 65211, Columbia, MO, USA

Dr. David H. Jonassen

College of Education and Human Development, University of North Dakota, 231 Centennial Drive, Stop 7189, 58202, Grand Forks, ND, USA

Dr. Woei Hung

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to David H. Jonassen .

Editor information

Editors and affiliations.

Faculty of Economics and Behavioral Sciences, Department of Education, University of Freiburg, 79085, Freiburg, Germany

Norbert M. Seel

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer Science+Business Media, LLC

About this entry

Cite this entry.

Jonassen, D.H., Hung, W. (2012). Problem Solving. In: Seel, N.M. (eds) Encyclopedia of the Sciences of Learning. Springer, Boston, MA. https://doi.org/10.1007/978-1-4419-1428-6_208

Download citation

DOI : https://doi.org/10.1007/978-1-4419-1428-6_208

Publisher Name : Springer, Boston, MA

Print ISBN : 978-1-4419-1427-9

Online ISBN : 978-1-4419-1428-6

eBook Packages : Humanities, Social Sciences and Law

Share this entry

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Help | Advanced Search

Computer Science > Computation and Language

Title: scibench: evaluating college-level scientific problem-solving abilities of large language models.

Abstract: Most of the existing Large Language Model (LLM) benchmarks on scientific problem reasoning focus on problems grounded in high-school subjects and are confined to elementary algebraic operations. To systematically examine the reasoning capabilities required for solving complex scientific problems, we introduce an expansive benchmark suite SciBench for LLMs. SciBench contains a carefully curated dataset featuring a range of collegiate-level scientific problems from mathematics, chemistry, and physics domains. Based on the dataset, we conduct an in-depth benchmarking study of representative open-source and proprietary LLMs with various prompting strategies. The results reveal that the current LLMs fall short of delivering satisfactory performance, with the best overall score of merely 43.22%. Furthermore, through a detailed user study, we categorize the errors made by LLMs into ten problem-solving abilities. Our analysis indicates that no single prompting strategy significantly outperforms the others and some strategies that demonstrate improvements in certain problem-solving skills could result in declines in other skills. We envision that SciBench will catalyze further developments in the reasoning abilities of LLMs, thereby ultimately contributing to scientific research and discovery.

Submission history

Access paper:.

• HTML (experimental)
• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

Integrative Oceanography Division

IOD distinguishes itself as scientific home for researchers working at the boundaries of traditional academic spheres, and generating growing programs in the integration of research with informatics as well as with education and public outreach.

Learn more about IOD

May 21, 2024

Scripps Student Spotlight: Lena Stasiak

May 14, 2024

UC San Diego-Led Science Teams Selected as Finalists for NASA Science Missions to Understand Our Changing Climate

May 08, 2024

Scripps Student Spotlight: Austin Barnes

• UCSD TV Perspectives on Ocean Science Lectures
• Scripps Collections Teaching and Outreach
• Birch Aquarium at Scripps Education Programs
• Birch Aquarium at Scripps Public Programs
• Scripps Technical Forum

No events are currently scheduled.

Directories & Administration

• Research Profiles
• Business Office
• Resources and Facilities

Related Resources

• California Current Ecosystem - LTER
• California Cooperative Oceanic Fisheries Investigations (CalCOFI)

Sign Up For Explorations Now