The decision is important.
You are trying to maximize your outcomes.
This section is adapted from:
Chapter 8: Make Good Decisions in Human Relations by Saylor Academy under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 License
Amabile, T. M. (1988). A model of creativity and innovation in organizations. In B. M. Staw & L. L. Cummings (Eds.), Research in organizational behavior (vol. 10, pp. 123–67). JAI Press.
Amabile, T. M., Conti, R., Coon, H., Lazenby, J., & Herron, M. (1996). Assessing the work environment for creativity. Academy of Management Journal , 39 , 1154–84.
Breen, B. (2000, August). What’s your intuition? Fast Company , 290.
Burke, L. A., & Miller, M. K. (1999). Taking the mystery out of intuitive decision making. Academy of Management Executive , 13 , 91–98.
Feist, G. J. (1998). A meta-analysis of personality in scientific and artistic creativity. Personality and Social Psychology Review , 2 , 290–309.
Ford, C. M., & Gioia, D. A. (2000). Factors influencing creativity in the domain of managerial decision making. Journal of Management , 26 , 705–32
Klein, G. (2003). Intuition at work . Doubleday.
Pauling, L. (n.d.). The best way to have good ideas. What Quote. Retrieved May 1, 2008, from http://www.whatquote.com/quotes/linus-pauling/250801-the-best-way-to-have.htm
Scott, G., Leritz, S, G , & Mumford, M. D. (2004). The effectiveness of creativity training: A quantitative review. Creativity Research Journal , 16 , 361–88.
Nutt, P. C. (1994). Types of organizational decision processes. Administrative Science Quarterly , 29 , 414–550.
Nutt, P. C. (1998). Surprising but true: Half the decisions in organizations fail. Academy of Management Executive , 13 , 75–90.
Salas, E., & Klein, G. (2001). Linking expertise and naturalistic decision making . Lawrence Erlbaum Associates.
Tierney, P., Farmer, S. M., & Graen, G. B. (1999). An examination of leadership and employee creativity: The relevance of traits and relationships. Personnel Psychology , 52 , 591–620.
Woodman, R. W., Sawyer, J. E., & Griffin, R. W. (1993). Toward a theory of organizational creativity. Academy of Management Review , 18 , 293–321.
Zell, D. M., Glassman, A. M., & Duron, S. A. (2007). Strategic management in turbulent times: The short and glorious history of accelerated decision making at Hewlett-Packard. Organizational Dynamics , 36 , 93–104.
Psychology, Communication, and the Canadian Workplace Copyright © 2022 by Laura Westmaas, BA, MSc is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.
The decision making process is a method of gathering information, assessing alternatives, and making a final choice with the goal of making the best decision possible. In this article, we detail the step-by-step process on how to make a good decision and explain different decision making methodologies.
We make decisions every day. Take the bus to work or call a car? Chocolate or vanilla ice cream? Whole milk or two percent?
There's an entire process that goes into making those tiny decisions, and while these are simple, easy choices, how do we end up making more challenging decisions?
At work, decisions aren't as simple as choosing what kind of milk you want in your latte in the morning. That’s why understanding the decision making process is so important.
The decision making process is the method of gathering information, assessing alternatives, and, ultimately, making a final choice.
In this ebook, learn how to equip employees to make better decisions—so your business can pivot, adapt, and tackle challenges more effectively than your competition.
Step 1: identify the decision that needs to be made.
When you're identifying the decision, ask yourself a few questions:
What is the problem that needs to be solved?
What is the goal you plan to achieve by implementing this decision?
How will you measure success?
These questions are all common goal setting techniques that will ultimately help you come up with possible solutions. When the problem is clearly defined, you then have more information to come up with the best decision to solve the problem.
Gathering information related to the decision being made is an important step to making an informed decision. Does your team have any historical data as it relates to this issue? Has anybody attempted to solve this problem before?
It's also important to look for information outside of your team or company. Effective decision making requires information from many different sources. Find external resources, whether it’s doing market research, working with a consultant, or talking with colleagues at a different company who have relevant experience. Gathering information helps your team identify different solutions to your problem.
This step requires you to look for many different solutions for the problem at hand. Finding more than one possible alternative is important when it comes to business decision-making, because different stakeholders may have different needs depending on their role. For example, if a company is looking for a work management tool, the design team may have different needs than a development team. Choosing only one solution right off the bat might not be the right course of action.
This is when you take all of the different solutions you’ve come up with and analyze how they would address your initial problem. Your team begins identifying the pros and cons of each option, and eliminating alternatives from those choices.
There are a few common ways your team can analyze and weigh the evidence of options:
Pros and cons list
SWOT analysis
Decision matrix
The next step is to make your final decision. Consider all of the information you've collected and how this decision may affect each stakeholder.
Sometimes the right decision is not one of the alternatives, but a blend of a few different alternatives. Effective decision-making involves creative problem solving and thinking out of the box, so don't limit you or your teams to clear-cut options.
One of the key values at Asana is to reject false tradeoffs. Choosing just one decision can mean losing benefits in others. If you can, try and find options that go beyond just the alternatives presented.
Once the final decision maker gives the green light, it's time to put the solution into action. Take the time to create an implementation plan so that your team is on the same page for next steps. Then it’s time to put your plan into action and monitor progress to determine whether or not this decision was a good one.
Once you’ve made a decision, you can monitor the success metrics you outlined in step 1. This is how you determine whether or not this solution meets your team's criteria of success.
Here are a few questions to consider when reviewing your decision:
Did it solve the problem your team identified in step 1?
Did this decision impact your team in a positive or negative way?
Which stakeholders benefited from this decision? Which stakeholders were impacted negatively?
If this solution was not the best alternative, your team might benefit from using an iterative form of project management. This enables your team to quickly adapt to changes, and make the best decisions with the resources they have.
While most decision making models revolve around the same seven steps, here are a few different methodologies to help you make a good decision.
This type of decision making model is the most common type that you'll see. It's logical and sequential. The seven steps listed above are an example of the rational decision making model.
When your decision has a big impact on your team and you need to maximize outcomes, this is the type of decision making process you should use. It requires you to consider a wide range of viewpoints with little bias so you can make the best decision possible.
This type of decision making model is dictated not by information or data, but by gut instincts. This form of decision making requires previous experience and pattern recognition to form strong instincts.
This type of decision making is often made by decision makers who have a lot of experience with similar kinds of problems. They have already had proven success with the solution they're looking to implement.
The creative decision making model involves collecting information and insights about a problem and coming up with potential ideas for a solution, similar to the rational decision making model.
The difference here is that instead of identifying the pros and cons of each alternative, the decision maker enters a period in which they try not to actively think about the solution at all. The goal is to have their subconscious take over and lead them to the right decision, similar to the intuitive decision making model.
This situation is best used in an iterative process so that teams can test their solutions and adapt as things change.
Tracking key decisions can be challenging when not documented correctly. Learn more about how a work management tool like Asana can help your team track key decisions, collaborate with teammates, and stay on top of progress all in one place.
Satisficing and bounded rationality, intra-organizational political decision making.
Our editors will review what you’ve submitted and determine whether to revise the article.
decision making , process and logic through which individuals arrive at a decision. Different models of decision making lead to dramatically different analyses and predictions. Decision-making theories range from objective rational decision making, which assumes that individuals will make the same decisions given the same information and preferences, to the more subjective logic of appropriateness , which assumes that specific institutional and organizational contexts matter in the decisions that individuals make.
(Read Steven Pinker’s Britannica entry on rationality.)
In modern Western societies the most common understanding of decision making is that it is rational—self-interested, purposeful, and efficient. During rational decision making, individuals will survey alternatives , evaluate consequences from each alternative , and finally do what they believe has the best consequences for themselves. The keys to a decision are the quality of information about alternatives and individual preferences. Modern economics is built on this understanding of how individuals make decisions.
Rational decision making becomes efficient when information is maximized and preferences are satisfied using the minimum of resources. In modern societies, rational decision making can occur in markets or firms. Both assume that individuals will act rationally, maximizing self-interest, but each works most efficiently under different conditions. Markets are most efficient when both buyers and sellers exist, when products or services are discrete so that the exchange can be one-time, when information about a product or service (such as its technology or means of evaluation) is broadly understood, and when there are enforced penalties for cheating.
Lacking these conditions, consensual exchange cannot occur, and rational individuals will try to cheat others to maximize their gain. In these cases a hierarchical organization is more efficient. The German sociologist Max Weber described how factories and bureaucracies became dramatically more efficient through growing technical expertise and, more importantly, a new division of labour , which divided work, specialized expertise, and coordinated individuals in a rule-based hierarchy . Bureaucracies decomposed complex technologies into manageable pieces, then allowed individuals to specialize and master a defined skill set. Using a clear hierarchy in which each position is controlled and supervised according to a stable and nonarbitrary system of rules, each individual’s work and expertise could be coordinated to achieve organizational goals, ranging from winning wars to making dresses.
In the 1940s, organization theorists began to challenge two assumptions necessary for rational decision making to occur, both of which were made obvious in cases where markets failed and hierarchies were necessary. First, information is never perfect, and individuals always make decisions based on imperfect information. Second, individuals do not evaluate all possible alternatives before making a choice. This behaviour is directly related to the costs of gathering information, because information becomes progressively more difficult and costly to gather. Instead of choosing the best alternative possible, individuals actually choose the first satisfactory alternative they find. The American social scientist Herbert Simon labeled this process “ satisficing” and concluded that human decision making could at best exhibit bounded rationality. Although objective rationality leads to only one possible rational conclusion, satisficing can lead to many rational conclusions, depending upon the information available and the imagination of the decision maker.
Simon argued that otherwise irrational individuals can behave rationally in the right context , particularly within a formal organization . Organizations can structure, or bound, individuals’ decisions by manipulating the premises on which decisions are made. Organizations can filter or emphasize information, bringing facts to an individual’s attention and identifying certain facts as important and legitimate . Individuals in hierarchies can take most of what happens around them for granted, concentrating only on a few key decisions. Hierarchies are efficient because they ensure that the correct information gets to the correct decision makers and that the correct person is making the decisions. At the same time, hierarchical organizations can socialize individuals to refrain from cheating by creating value decision premises that underlie decision makers’ judgments on what is right or good to do. These values, beliefs, or norms can come from family, from school, or from within the organization, but the organization can structure environments so that the most desirable value will be most salient at the time of decision.
Hierarchical organizations can structure factual and value decision premises so that the range of action becomes so narrow that only one alternative remains: the rational choice. Structuring decision premises can be done by directly managing information, selectively recruiting members, training members, and creating closed promotion patterns.
Organizations become rational in pursuing their missions through what Simon called ends-means chains. Leaders set the organizational mission, find a set of means for achieving the mission, take each of those means as a subgoal, and then find means for the subgoals and so on, until goals exist for every member of the organization. Leaders thus create a hierarchy of goals, in which each organizational level’s goals are an end relative to the levels below it and a means relative to the levels above it. Each individual’s work thus becomes a small part of accomplishing the organization’s mission.
Turning Simon’s bounded rationality on its head, other theorists argued that organizations are not purposeful cohesive actors but rather groups of competing coalitions made up of individuals with disparate interests. Individuals do not represent organizational interests; organizations represent individuals’ interests. Seen from this perspective, it is erroneous to ascribe a mission to an organization. Instead, organizations have goals set by a temporarily dominant coalition, which itself has no permanent goals and whose membership is subject to change. Members of the dominant coalition make decisions by bargaining, negotiating, and making side payments. Organizational decision making is the product of the game rather than a rational, goal-oriented process. Individual decision making is rational in the narrow sense that individuals pursue individual, self-interested goals, though this cannot always be accomplished directly. Individuals must pick their fights and use their influence carefully.
To understand and possibly predict what organizations will do, it is necessary to uncover and analyze the membership of the dominant coalition . The formal organizational chart is not a reliable map of organizational power. Instead, analysts must discover authority. Individuals gain authority by being able to resolve uncertainty. Individuals that can unravel technical problems, attract resources, or manage internal conflict demonstrate their usefulness to the rest of the organization and gain power. Working in concert with others who can perform similarly valuable functions, they become part of the dominant coalition. The size and composition of the dominant coalition depend on the types of environmental, technical, or coordinating uncertainty that must be resolved for the organization to survive. More technically complex, larger organizations in rapidly changing environments will tend to have larger dominant coalitions.
Discover the powerful 7-Step Problem-Solving Process to make better decisions and achieve better outcomes. Master the art of problem-solving in this comprehensive guide. Download the Free PowerPoint and PDF Template.
Introduction.
The 7-Step Problem-Solving Process involves steps that guide you through the problem-solving process. The first step is to define the problem, followed by disaggregating the problem into smaller, more manageable parts. Next, you prioritize the features and create a work plan to address each. Then, you analyze each piece, synthesize the information, and communicate your findings to others.
In this article, we'll explore each step of the 7-Step Problem-Solving Process in detail so you can start mastering this valuable skill. At the end of the blog post, you can download the process's free PowerPoint and PDF templates .
One way to define the problem is to ask the right questions. Questions like "What is the problem?" and "What are the causes of the problem?" can help. Gathering data and information about the issue to assist in the definition process is also essential.
After defining the problem, the next step in the 7-step problem-solving process is to disaggregate the problem into smaller, more manageable parts. Disaggregation helps break down the problem into smaller pieces that can be analyzed individually. This step is crucial in understanding the root cause of the problem and identifying the most effective solutions.
Disaggregation helps in breaking down complex problems into smaller, more manageable parts. It helps understand the relationships between different factors contributing to the problem and identify the most critical factors that must be addressed. By disaggregating the problem, decision-makers can focus on the most vital areas, leading to more effective solutions.
Once the issues have been prioritized, developing a plan of action to address them is essential. This involves identifying the resources required, setting timelines, and assigning responsibilities.
The work plan should include a list of tasks, deadlines, and responsibilities for each team member involved in the problem-solving process. Assigning tasks based on each team member's strengths and expertise ensures the work is completed efficiently and effectively.
Developing a work plan is a critical step in the problem-solving process. It provides a clear roadmap for solving the problem and ensures everyone involved is aligned and working towards the same goal.
Pareto analysis is another method that can be used during the analysis phase. This method involves identifying the 20% of causes responsible for 80% of the problems. By focusing on these critical causes, organizations can make significant improvements.
Once the analysis phase is complete, it is time to synthesize the information gathered to arrive at a solution. During this step, the focus is on identifying the most viable solution that addresses the problem. This involves examining and combining the analysis results for a clear and concise conclusion.
During the synthesis phase, it is vital to remain open-minded and consider all potential solutions. Involving all stakeholders in the decision-making process is essential to ensure everyone's perspectives are considered.
In addition to the report, a presentation explaining the findings is essential. The presentation should be tailored to the audience and highlight the report's key points. Visual aids such as tables, graphs, and charts can make the presentation more engaging.
The 7-step problem-solving process is a powerful tool for helping individuals and organizations make better decisions. By following these steps, individuals can identify the root cause of a problem, prioritize potential solutions, and develop a clear plan of action. This process can be applied to various scenarios, from personal challenges to complex business problems.
By mastering the 7-step problem-solving process, individuals can become more effective decision-makers and problem-solvers. This process can help individuals and organizations save time and resources while improving outcomes. With practice, individuals can develop the skills to apply this process to a wide range of scenarios and make better decisions in all areas of life.
Free powerpoint and pdf template, executive summary: the 7-step problem-solving process.
Mastering this process can improve decision-making and problem-solving capabilities, save time and resources, and improve outcomes in personal and professional contexts.
I'd appreciate your support if my templates have saved you time or helped you start a project. Buy Me a Coffee is a simple way to show your appreciation and help me continue creating high-quality templates that meet your needs.
7-step problem-solving process powerpoint template, multi-chapter growth strategy framework (free template), lidl swot analysis: free ppt template and in-depth insights.
Discover Lidl's strengths, weaknesses, opportunities, and threats with our free PowerPoint template. This in-depth SWOT analysis provides valuable insights to help you understand Lidl's market position and strategic direction.
Pestle analysis: decoding reddit's landscape (free ppt).
Intended for healthcare professionals
This is the fourth in a series of five articles
This article reviews our current understanding of the cognitive processes involved in diagnostic reasoning in clinical medicine. It describes and analyses the psychological processes employed in identifying and solving diagnostic problems and reviews errors and pitfalls in diagnostic reasoning in the light of two particularly influential approaches: problem solving 1 , 2 , 3 and decision making. 4 , 5 , 6 , 7 , 8 Problem solving research was initially aimed at describing reasoning by expert physicians, to improve instruction of medical students and house officers. Psychological decision research has been influenced from the start by statistical models of reasoning under uncertainty, and has concentrated on identifying departures from these standards.
Problem solving and decision making are two paradigms for psychological research on clinical reasoning, each with its own assumptions and methods
The choice of strategy for diagnostic problem solving depends on the perceived difficulty of the case and on knowledge of content as well as strategy
Final conclusions should depend both on prior belief and strength of the evidence
Conclusions reached by Bayes's theorem and clinical intuition may conflict
Because of cognitive limitations, systematic biases and errors result from employing simpler rather than more complex cognitive strategies
Evidence based medicine applies decision theory to clinical diagnosis
Diagnosis as selecting a hypothesis.
The earliest psychological formulation viewed diagnostic reasoning as a process of testing hypotheses. Solutions to difficult diagnostic problems were found by generating a limited number of hypotheses early in the diagnostic process and using them to guide subsequent collection of data. 1 Each hypothesis can be used to predict what additional findings ought to be present if it were true, and the diagnostic process is a guided search for these findings. Experienced physicians form hypotheses and their diagnostic plan rapidly, and the …
BMA Member Log In
If you have a subscription to The BMJ, log in:
Subscribe from £184 *.
Subscribe and get access to all BMJ articles, and much more.
* For online subscription
Access this article for 1 day for: £50 / $60/ €56 ( excludes VAT )
You can download a PDF version for your personal record.
Buy this article
Harvard Business School Online's Business Insights Blog provides the career insights you need to achieve your goals and gain confidence in your business skills.
One of the biggest hindrances to innovation is complacency—it can be more comfortable to do what you know than venture into the unknown. Business leaders can overcome this barrier by mobilizing creative team members and providing space to innovate.
There are several tools you can use to encourage creativity in the workplace. Creative problem-solving is one of them, which facilitates the development of innovative solutions to difficult problems.
Here’s an overview of creative problem-solving and why it’s important in business.
Access your free e-book today.
Research is necessary when solving a problem. But there are situations where a problem’s specific cause is difficult to pinpoint. This can occur when there’s not enough time to narrow down the problem’s source or there are differing opinions about its root cause.
In such cases, you can use creative problem-solving , which allows you to explore potential solutions regardless of whether a problem has been defined.
Creative problem-solving is less structured than other innovation processes and encourages exploring open-ended solutions. It also focuses on developing new perspectives and fostering creativity in the workplace . Its benefits include:
Creative problem-solving is traditionally based on the following key principles :
Creative problem-solving uses two primary tools to find solutions: divergence and convergence. Divergence generates ideas in response to a problem, while convergence narrows them down to a shortlist. It balances these two practices and turns ideas into concrete solutions.
By framing problems as questions, you shift from focusing on obstacles to solutions. This provides the freedom to brainstorm potential ideas.
When brainstorming, it can be natural to reject or accept ideas right away. Yet, immediate judgments interfere with the idea generation process. Even ideas that seem implausible can turn into outstanding innovations upon further exploration and development.
Using negative words like "no" discourages creative thinking. Instead, use positive language to build and maintain an environment that fosters the development of creative and innovative ideas.
Whereas creative problem-solving facilitates developing innovative ideas through a less structured workflow, design thinking takes a far more organized approach.
Design thinking is a human-centered, solutions-based process that fosters the ideation and development of solutions. In the online course Design Thinking and Innovation , Harvard Business School Dean Srikant Datar leverages a four-phase framework to explain design thinking.
The four stages are:
Creative problem-solving primarily operates in the ideate phase of design thinking but can be applied to others. This is because design thinking is an iterative process that moves between the stages as ideas are generated and pursued. This is normal and encouraged, as innovation requires exploring multiple ideas.
While there are many useful tools in the creative problem-solving process, here are three you should know:
One way to innovate is by creating a story about a problem to understand how it affects users and what solutions best fit their needs. Here are the steps you need to take to use this tool properly.
Create a problem story to identify the undesired phenomena (UDP). For example, consider a company that produces printers that overheat. In this case, the UDP is "our printers overheat."
To move forward in time, ask: “Why is this a problem?” For example, minor damage could be one result of the machines overheating. In more extreme cases, printers may catch fire. Don't be afraid to create multiple problem stories if you think of more than one UDP.
To move backward in time, ask: “What caused this UDP?” If you can't identify the root problem, think about what typically causes the UDP to occur. For the overheating printers, overuse could be a cause.
Following the three-step framework above helps illustrate a clear problem story:
You can extend the problem story in either direction if you think of additional cause-and-effect relationships.
By this point, you’ll have multiple UDP storylines. Take two that are similar and focus on breaking the chains connecting them. This can be accomplished through inversion or neutralization.
Even if creating a problem story doesn't provide a solution, it can offer useful context to users’ problems and additional ideas to be explored. Given that divergence is one of the fundamental practices of creative problem-solving, it’s a good idea to incorporate it into each tool you use.
Brainstorming is a tool that can be highly effective when guided by the iterative qualities of the design thinking process. It involves openly discussing and debating ideas and topics in a group setting. This facilitates idea generation and exploration as different team members consider the same concept from multiple perspectives.
Hosting brainstorming sessions can result in problems, such as groupthink or social loafing. To combat this, leverage a three-step brainstorming method involving divergence and convergence :
The alternate worlds tool is an empathetic approach to creative problem-solving. It encourages you to consider how someone in another world would approach your situation.
For example, if you’re concerned that the printers you produce overheat and catch fire, consider how a different industry would approach the problem. How would an automotive expert solve it? How would a firefighter?
Be creative as you consider and research alternate worlds. The purpose is not to nail down a solution right away but to continue the ideation process through diverging and exploring ideas.
Whether you’re an entrepreneur, marketer, or business leader, learning the ropes of design thinking can be an effective way to build your skills and foster creativity and innovation in any setting.
If you're ready to develop your design thinking and creative problem-solving skills, explore Design Thinking and Innovation , one of our online entrepreneurship and innovation courses. If you aren't sure which course is the right fit, download our free course flowchart to determine which best aligns with your goals.
13 Accesses
Explore all metrics
The presence of observation and action delays in remote control scenarios significantly challenges the decision-making of agents that depend on immediate interactions, particularly within traditional deep reinforcement learning (DRL) algorithms. Existing approaches attempt to tackle this problem through various strategies, such as predicting delayed states, transforming delayed Markov Decision Processes (MDPs) into delay-free equivalents. However, both model-free and model-based methods require extensive online data, making them time-consuming and resource-intensive. To effectively handle time-delay challenges and develop a competent and robust RL algorithm, the Augmented Decision Transformer (ADT) is proposed as the first offline RL algorithm designed to enable agents to manage diverse tasks with various constant delays. It transforms a deterministic delayed MDP (DDMDP) into a standard MDP by simulating trajectories in delayed environments using offline dataset from undelayed environments. The Decision Transformer, an autoregressive model, is then employed to train a decision model based on expected rewards, past state sequences and past action sequences. Extensive experiments conducted on MuJoCo and Adroit tasks validate the robustness and efficiency of the ADT, with its average performance across all tasks being 56% better than the worst-performing comparative algorithms. The results demonstrate that the ADT can outperform state-of-the-art RL counterparts, achieving superior performance across various tasks with different delay conditions.
This is a preview of subscription content, log in via an institution to check access.
Subscribe and save.
Price includes VAT (Russian Federation)
Instant access to the full article PDF.
Rent this article via DeepDyve
Institutional subscriptions
Explore related subjects.
Data will be made available on reasonable request.
Code will be made available on reasonable request.
\(\delta (x - x_0) = {\left\{ \begin{array}{ll} 1, & \text {if }x=x_0, \\ {0}, & \text {others.} \end{array}\right. }\)
\(c_i\) represents the action selected at time i before the first state is observed.
https://github.com/Farama-Foundation/D4RL
https://mujoco.org/
https://github.com/aravindr93/hand_dapg
https://gym.openai.com/
https://github.com/young-geng/CQL
https://github.com/sfujim/TD3_BC
https://github.com/rmst/rlrd
https://github.com/baimingc/dambrl
https://github.com/pranz24/pytorch-soft-actor-critic
van Dis EA, Bollen J, Zuidema W et al (2023) Chatgpt: five priorities for research. Nature 614(7947):224–226
Degrave J, Felici F, Buchli J et al (2022) Magnetic control of tokamak plasmas through deep reinforcement learning. Nature 602(7897):414–419
Article Google Scholar
Zhang R, Lv Q, Li J et al (2022) A reinforcement learning method for human-robot collaboration in assembly tasks. Robot Comput Integr Manuf 73:102227
Jiang W, Liu K, Charalambous T (2022) Multi-agent consensus with heterogeneous time-varying input and communication delays in digraphs. Automatica 135:109950
Article MathSciNet Google Scholar
Jiang W, Chen Y, Charalambous T (2020) Consensus of general linear multi-agent systems with heterogeneous input and communication delays. IEEE Control Systems Letters 5(3):851–856
Chen H, Liu Z (2021) Time-delay prediction-based smith predictive control for space teleoperation. J Guid Control Dyn 44(4):872–879
Guerrero J, Chemori A, Torres J et al (2023) Time-delay high-order sliding mode control for trajectory tracking of autonomous underwater vehicles under disturbances. Ocean Eng 268:113375
Liu H, Wang L (2020) Remote human-robot collaboration: A cyber-physical system application for hazard manufacturing environment. J Manuf Syst 54:24–34
Zhou X, Yang Z, Ren Y et al (2023) Modified bilateral active estimation model: A learning-based solution to the time delay problem in robotic tele-control. IEEE Robot Autom Lett 8(5):2653–2660
Haarnoja T, Zhou A, Abbeel P et al (2018) Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In: International conference on machine learning, PMLR, pp 1861–1870
Deng Y, Léchappé V, Moulay E et al (2022) Predictor-based control of time-delay systems: a survey. Int J Syst Sci 53(12):2496–2534
Zhang XM, Han QL, Ge X (2022) The construction of augmented lyapunov-krasovskii functionals and the estimation of their derivatives in stability analysis of time-delay systems: A survey. Int J Syst Sci 53(12):2480–2495
Ali A, Zhu Y, Zakarya M (2022) Exploiting dynamic spatio-temporal graph convolutional neural networks for citywide traffic flows prediction. Neural Netw 145:233–247
Ali A, Zhu Y, Zakarya M (2021) Exploiting dynamic spatio-temporal correlations for citywide traffic flow prediction using attention based neural networks. Inf Sci 577:852–870
Ali A, Zhu Y, Zakarya M (2021) A data aggregation based approach to exploit dynamic spatio-temporal correlations for citywide crowd flows prediction in fog computing. Multimed Tools Appl 80(20):31401–31433
Walsh TJ, Nouri A, Li L et al (2009) Learning and planning in environments with delayed feedback. Auton Agent Multi-Agent Syst 18:83–105
Hester T, Stone P (2013) Texplore: real-time sample-efficient reinforcement learning for robots. Mach Learn 90:385–429
Derman E, Dalal G, Mannor S (2020) Acting in delayed environments with non-stationary markov policies. In: International conference on learning representations
Katsikopoulos KV, Engelbrecht SE (2003) Markov decision processes with delays and asynchronous cost collection. IEEE Trans Autom Control 48(4):568–574
Nath S, Baranwal M, Khadilkar H (2021) Revisiting state augmentation methods for reinforcement learning with stochastic delays. In: Proceedings of the 30th ACM international conference on information & knowledge management, pp 1346–1355
Ramstedt S, Pal C (2019) Real-time reinforcement learning. Adv Neural Inf Process Syst 32
Xiao T, Jang E, Kalashnikov D et al (2019) Thinking while moving: Deep reinforcement learning with concurrent control. In: International conference on learning representations
Bouteiller Y, Ramstedt S, Beltrame G et al (2021) Reinforcement learning with random delays. In: International conference on learning representations
Chen L, Lu K, Rajeswaran A et al (2021) Decision transformer: Reinforcement learning via sequence modeling. Adv Neural Inf Process Syst 34:15084–15097
Google Scholar
Schuitema E, Buşoniu L, Babuška R et al (2010) Control delay in reinforcement learning for real-time dynamic systems: A memoryless approach. In: 2010 IEEE/RSJ International conference on intelligent robots and systems, IEEE, pp 3226–3231
Agarwal M, Aggarwal V (2021) Blind decision making: Reinforcement learning with delayed observations. Pattern Recogn Lett 150:176–182
Chen B, Xu M, Li L et al (2021) Delay-aware model-based reinforcement learning for continuous control. Neurocomputing 450:119–128
Prudencio RF, Maximo MR, Colombini EL (2023) A survey on offline reinforcement learning: Taxonomy, review, and open problems. IEEE Trans Neural Netw Learn Syst
Fujimoto S, Meger D, Precup D (2019) Off-policy deep reinforcement learning without exploration. In: International conference on machine learning, PMLR, pp 2052–2062
Kumar A, Fu J, Soh M et al (2019) Stabilizing off-policy q-learning via bootstrapping error reduction. Adv Neural Inf Process Syst 32
Fujimoto S, Gu SS (2021) A minimalist approach to offline reinforcement learning. Adv Neural Inf Process Syst 34:20132–20145
Kumar A, Zhou A, Tucker G et al (2020) Conservative q-learning for offline reinforcement learning. Adv Neural Inf Process Syst 33:1179–1191
Yu T, Kumar A, Rafailov R et al (2021) Combo: Conservative offline model-based policy optimization. Adv Neural Inf Process Syst 34:28954–28967
Janner M, Fu J, Zhang M et al (2019) When to trust your model: Model-based policy optimization. Adv Neural Inf Process Syst 32
Kidambi R, Rajeswaran A, Netrapalli P et al (2020) Morel: Model-based offline reinforcement learning. Adv Neural Inf Process Syst 33:21810–21823
Zhang R, Dai B, Li L et al (2020) Gendice: Generalized offline estimation of stationary values. Int Conf Learn Representations
Janner M, Li Q, Levine S (2021) Offline reinforcement learning as one big sequence modeling problem. Adv Neural Inf Process Syst 34:1273–1286
Li W, Luo H, Lin Z et al (2023) A survey on transformers in reinforcement learning. Transactions on Machine Learning Research pp 2835–8856
Vinyals O, Babuschkin I, Czarnecki WM et al (2019) Grandmaster level in starcraft ii using multi-agent reinforcement learning. Nature 575(7782):350–354
Parisotto E, Song F, Rae J, et al (2020) Stabilizing transformers for reinforcement learning. In: International conference on machine learning, PMLR, pp 7487–7498
Micheli V, Alonso E, Fleuret F (2023) Transformers are sample efficient world models
Ozair S, Li Y, Razavi A et al (2021) Vector quantized models for planning. In: international conference on machine learning, PMLR, pp 8302–8313
Paster K, McIlraith S, Ba J (2022) You can’t count on luck: Why decision transformers and rvs fail in stochastic environments. Adv Neural Inf Process Syst 35:38966–38979
Yang M, Schuurmans D, Abbeel P et al (2023) Dichotomy of control: Separating what you can control from what you cannot. The Eleventh International Conference on Learning Representations
Lee KH, Nachum O, Yang M et al (2022) Multi-game decision transformers. Adv Neural Inf Process Syst
Sutton RS, Barto AG (2018) Reinforcement learning: An introduction. MIT press
Hu K, Zheng RC, Gao Y et al (2023) Decision transformer under random frame dropping. The Eleventh International Conference on Learning Representations
Download references
Not applicable.
Authors and affiliations.
Shenzhen International Graduate School, Tsinghua University, Shenzhen, 518055, China
Bo Xia, Zaihui Yang, Minzhi Xie, Yongzhe Chang, Zhiheng Li & Xueqian Wang
Research Institute of Tsinghua University in Shenzhen, Shenzhen, 518057, China
Department of Automation, Tsinghua University, Beijing, 100084, China
You can also search for this author in PubMed Google Scholar
− Conceptualization: Bo Xia and Xueqian Wang− Methodology: Bo Xia and Xueqian Wang− Formal analysis and investigation: Bo Xia, Zaihui Yang, and Minzhi Xie− Writing - original draft preparation: Bo Xia, Minzhi Xie, Bo Yuan, and Yongzhe Chang− Writing - review and editing: Bo Xia, Bo Yuan, Zhiheng Li, and Xueqian Wang− Supervision: Yongzhe Chang, Bo Yuan, Zhiheng Li, Xueqian Wang, and Bin Liang− Funding acquisition: Xueqian Wang and Bin Liang
Correspondence to Xueqian Wang .
Ethics approval and consent to participate, consent for publication, competing interests.
All authors disclosed no relevant relationships.
Publisher's note.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
For offline RL algorithms, including ADT, CQL, TD3_BC, and BC, the expert rewards for various tasks are sourced from D4RL. For the RLRD and SAC algorithms, the expert rewards for MuJoCo tasks are obtained from Tianshou, while the expert rewards for Adroit tasks (including pen, door, relocate, and hammer) are derived from the average experimental results of three different seeds. For the DATS algorithm, as reward functions for Adroit tasks are not designed, only MuJoCo tasks are considered, and the expert rewards are similarly obtained from the average of three different experiments.
Table 13 shows all the expert rewards in various environments with different algorithms.
Tables 14 through 21 present the average normalized returns from ten test runs under delays 1 to 3, for eight tasks in MuJoCo and Adroit environments, using expert and medium-expert datasets. Each column’s “Dataset” string indicates the delay and dataset type, e.g., “1-e” represents a delay of 1 using the expert dataset. The “Average” value in the last row of each table reflects the overall performance of each algorithm, averaged across all delays and datasets for easier comparison.
From these tables, the following conclusions can be drawn: (1) For most tasks, the performance generally declines as the delay increases when using the same dataset and algorithm in a delay-free environment. This is due to the increasing discrepancy between the observed state and the actual state, impacting decision quality. However, ADT outperforms other algorithms in most tasks and exhibits a slower performance decline with increasing delay. This is attributed to ADT’s trajectory optimization approach and reward-guided updates, leveraging the Transformer architecture to deeply explore information context. (2) The experimental results indicate that for the same task and delay, policies trained on the expert dataset generally outperform those trained on the medium-expert dataset. This suggests that having more expert-level trajectories in the dataset, given a similar number of trajectories, is beneficial for policy learning. (3) Adroit tasks present significant challenges for conventional offline reinforcement learning methods due to their large action space and sparse reward settings. Further research is required to address the policy learning problem with limited expert trajectories in such tasks.
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
Reprints and permissions
Xia, B., Yang, Z., Xie, M. et al. Solving time-delay issues in reinforcement learning via transformers. Appl Intell (2024). https://doi.org/10.1007/s10489-024-05830-2
Download citation
Accepted : 30 August 2024
Published : 10 September 2024
DOI : https://doi.org/10.1007/s10489-024-05830-2
Anyone you share the following link with will be able to read this content:
Sorry, a shareable link is not currently available for this article.
Provided by the Springer Nature SharedIt content-sharing initiative
IMAGES
VIDEO
COMMENTS
Here is a six-step process to follow when using a problem-solving model: 1. Define the problem. First, determine the problem that your team needs to solve. During this step, teams may encourage open and honest communication so everyone feels comfortable sharing their thoughts and concerns.
A decision-making model works by walking you through the decision-making process — and there are several such models available for you to choose from. To help you improve your problem-solving abilities and make better decisions, let's take a look at five proven decision-making models and when you should use them. Defining decision-making ...
These models provide a structured approach to problem-solving and decision-making by encouraging the evaluation of information and arguments in a logical, systematic manner. By understanding and applying these models, one can learn to make well-reasoned judgments and decisions.
7. Solution evaluation. 1. Problem identification. The first stage of any problem solving process is to identify the problem (s) you need to solve. This often looks like using group discussions and activities to help a group surface and effectively articulate the challenges they're facing and wish to resolve.
The chapter introduces principles of rational choice suggested by classical decision theory, followed by a discussion of research on human decision making which has led to the new perspectives of behavioral decision theory and behavioral economics, and naturalistic decision models. It addresses the topic of decision support and problem solving ...
Problem solving and decision making are both examples of complex, higher-order thinking. Both involve the assessment of the environment, the involvement of working memory or short-term memory, reliance on long term memory, effects of knowledge, and the application of heuristics to complete a behavior. A problem can be defined as an impasse or ...
Read more on Decision making and problem solving or related topics Leadership and managing people, Leadership development, Leadership qualities, Personal growth and transformation and Early career MM
Decision-making is perhaps best thought of as a key part of problem-solving: one part of the overall process. Our approach at Skills You Need is to set out a framework to help guide you through the decision-making process. You won't always need to use the whole framework, or even use it at all, but you may find it useful if you are a bit ...
Become a better problem solver with insights and advice from leaders around the world on topics including developing a problem-solving mindset, solving problems in uncertain times, problem solving with AI, and much more. ... decision-makers can't effectively act alone when it comes to solving complicated workplace problems; diverse ...
Iterative problem solving is a critical part of this. Sometimes, people think work planning sounds dull, but it isn't. It's how we know what's expected of us and when we need to deliver it and how we're progressing toward the answer. It's also the place where we can deal with biases. Bias is a feature of every human decision-making ...
The decision-making process is a seven-step framework for making a big decision with careful consideration, collaboration, and confidence. ... Most of us are eager to tie on our superhero capes and jump into problem-solving mode — especially if our team is depending on a solution. ... And the truth is that the model is likely overkill for the ...
Decision making and problem solving Magazine Article. Paul Rogers. Marcia W. Blenko. Your organization can become more decisive—and can implement strategy more quickly—if you know where the ...
Finding a suitable solution for issues can be accomplished by following the basic four-step problem-solving process and methodology outlined below. Step. Characteristics. 1. Define the problem. Differentiate fact from opinion. Specify underlying causes. Consult each faction involved for information. State the problem specifically.
7.2 Decision-making Models Decision-making Models. In this section, we are going to discuss different decision-making models designed to understand and evaluate the effectiveness of nonprogrammed decisions. ... All the models include problem identification, which is the step in which the need for problem solving becomes apparent. If you do not ...
Step 3: Identify alternative solutions. This step requires you to look for many different solutions for the problem at hand. Finding more than one possible alternative is important when it comes to business decision-making, because different stakeholders may have different needs depending on their role.
decision making, process and logic through which individuals arrive at a decision. Different models of decision making lead to dramatically different analyses and predictions. Decision-making theories range from objective rational decision making, which assumes that individuals will make the same decisions given the same information and ...
The FOCUS Model is a simple quality-improvement tool commonly used in the health care industry. You can use it to improve any process, but it is particularly useful for processes that span different departments. The five steps in FOCUS are as follows: F ind the problem. O rganize a team. C larify the problem.
A core managerial competency is a manager's ability to make sound decisions that solve problems. This article examines the nature and the process of managerial decision-making. In doing so, it identifies the activities comprising managerial decision-making and discusses common decision-making practices, including the often-used but limiting practice called the typology method of coping along ...
Decision-making is the process of choosing a solution based on your judgment, situation, facts, knowledge or a combination of available data. The goal is to avoid potential difficulties. Identifying opportunity is an important part of the decision-making process. Making decisions is often a part of problem-solving.
Step 1: Define the Problem. The first step in the problem-solving process is to define the problem. This step is crucial because finding a solution is only accessible if the problem is clearly defined. The problem must be specific, measurable, and achievable. One way to define the problem is to ask the right questions.
This is the fourth in a series of five articles This article reviews our current understanding of the cognitive processes involved in diagnostic reasoning in clinical medicine. It describes and analyses the psychological processes employed in identifying and solving diagnostic problems and reviews errors and pitfalls in diagnostic reasoning in the light of two particularly influential ...
Creative problem-solving primarily operates in the ideate phase of design thinking but can be applied to others. This is because design thinking is an iterative process that moves between the stages as ideas are generated and pursued. This is normal and encouraged, as innovation requires exploring multiple ideas.
The authors proposed a conceptual model of how people respond to offers of help, which was implemented as a computational model by using mathematical expressions to represent when, and how, these different social norms factor into the decision-making process. In a novel decision-making task, participants were subjected to electric shocks which ...
The presence of observation and action delays in remote control scenarios significantly challenges the decision-making of agents that depend on immediate interactions, particularly within traditional deep reinforcement learning (DRL) algorithms. Existing approaches attempt to tackle this problem through various strategies, such as predicting delayed states, transforming delayed Markov Decision ...