case study on various network applications

• Network infrastructure

Getty Images/iStockphoto

9 types of networks and their use cases

Networks differ based on size, connectivity, coverage and design. this guide explores the most common types of networks, their benefits and use cases..

• Deanna Darah, Site Editor

A computer network is an interconnected system of devices, represented as network nodes, that share information, data and resources among each other.

Network devices can be as simple as computers or smartphones that connect into a network. Larger networks use devices like routers and switches to create the underlying network infrastructure.

Not all networks are the same, however. Several types of networks are available, and each exists to support the devices, size and location of the system. Networks also have different levels of access and forms of connectivity.

Practicing and aspiring network professionals need to understand the different types of networks to monitor, manage and maintain their organization's chosen configuration. The size, scale and purpose of a network determine the design as well as let network professionals operate and manage the network at scale.

The following are common types of networks, along with their benefits, use cases and other essential information.

1. Personal area network

A personal area network (PAN) is the smallest and simplest type of network. PANs connect devices within the range of an individual and are no larger than about 10 meters (m). Because PANs operate in such limited areas of space, most are wireless and provide short-range connectivity with infrared technology.

An example of a wireless PAN is when users connect Bluetooth devices, like wireless headsets, to a smartphone or laptop. Although most PANs are wireless, wired PAN options are available, such as USB.

PAN benefits

• Portability. Most devices that connect in a PAN are small and easily transportable.
• Affordability. The ability to form a connection between two devices in a PAN without additional wiring is often less expensive than a wired network.
• Reliability. PANs guarantee stable connectivity between devices, provided that the devices remain within the 10 m range.
• Security. PANs don't connect directly to larger networks, such as the internet, but they can connect to other devices with access to larger networks. The security of a device in a PAN depends on how secure the intermediary device is, as this device acts as a gateway between the PAN and the broader network.

PAN use cases

• Personal network configurations. PANs let users connect devices within their vicinity, which establishes small personal networks. A literal example of this is a body area network, in which a user physically wears connected devices.
• Home networks. Small home networks with computers, printers and other wireless devices are considered PANs.
• IoT systems. PANs can optimize and support IoT systems, such as users with medical wearables or networks with smart devices, in offices and homes.

2. Local area network

A local area network ( LAN ) is a system in which computers and other devices connect to each other in one location. While PANs connect devices around an individual, the scope of a LAN can range from a few meters in a home to hundreds of meters in a large office. The network topology determines how devices in LANs interconnect, such as a ring or mesh topology.

LANs use both wired and wireless connectivity options . Wireless LAN (WLAN) has surpassed traditional wired LAN in popularity, but wired LAN remains the more secure and reliable option. Wired LANs use physical cables, such as Ethernet, and switches. WLANs use devices such as wireless routers and access points to interconnect network devices through radio frequency waves.

Wired LANs are usually more secure than WLANs because they require a physical cable to form a connection and are less susceptible to compromise. However, network administrators can implement security protocols and encryption standards to secure wireless networks .

LAN benefits

• Resource sharing. Resource sharing is one of the most important reasons to set up any network. As devices connect to each other, they can share more files, data and software among each other.
• Secure data storage. Network data resides in a centralized location that all connected devices can access. Devices must receive permission to access the network, which prevents unauthorized users from retrieving sensitive information.
• Fast communication. Ethernet cables provide fast, reliable data transmission speeds, which increases the rate of communication between devices.
• Seamless communication. Any authorized device can communicate with another on the same network.

LAN use cases

• Home office and corporate network connectivity. Users in personal home offices and office networks can connect their devices and transfer data between each device.
• Data sharing . Employees in company offices can quickly communicate, share and access the same data and services provided by their organizations.
• Wi-Fi access. Wi-Fi is the most common WLAN use case. A wireless network can use Wi-Fi radio signals to connect multiple devices in a single location. While WLAN and Wi-Fi might sound like similar technologies, they aren't the same. A Wi-Fi network is a WLAN, but not all WLANs use Wi-Fi.

Virtual LAN

A virtual LAN ( VLAN ) is a type of LAN configuration that virtually groups network components into segments. Network administrators create VLANs to operate segments as individual systems, separate from the rest of the LAN. VLANs prevent network congestion by isolating LAN traffic for each segment, which improves network performance and efficiency, simplifies network management and increases security.

3. Metropolitan area network

A metropolitan area network ( MAN ) is an interconnection of several LANs throughout a city, town or municipality. Like LANs, a MAN use various wired or wireless connectivity options, such as fiber optics, Ethernet cables, Wi-Fi or cellular.

MAN benefits

• Municipal coverage. A MAN can span an entire city or town, which provides network connectivity for dozens of miles.
• Efficient networking standards. MAN configurations typically use IEEE 802.11 networking standards to increase bandwidth capacity and frequency levels, which boost network performance.
• High-speed connectivity. Fiber optic cables are the most popular form of MAN connectivity because they provide reliable and fast connection data rates.

MAN use cases

• Extended network connectivity. MANs enable access to the same network in multiple locations. In a LAN, users can only access the network in one location. In a MAN, organizations with LANs in the same municipality, such as different office buildings, can extend their network to those areas.
• Community network access. Government entities might configure a MAN to provide public network connectivity to users. One example is when municipalities offer free, public Wi-Fi to city residents via wireless MAN technology.
• Smart city connectivity. MANs provide and enable connectivity in smart cities. They provide features like intelligent transportation systems, IoT deployment, smart grids and other city services.

4. Campus network

A campus network , sometimes referred to as a campus area network or CAN , is a network of interconnected, dispersed LANs. Like MANs, campus networks extend coverage to buildings close in proximity. However, campus networks connect LANs within a limited geographical area, while MANs connect LANs across a larger metro area. A MAN can extend to 50 kilometers, but the geographical range of a campus network varies from 1 km to 5 km.

Campus benefits

• Affordability. Campus networks cover a smaller geographical area than MANs, so infrastructure is less costly to maintain.
• Easy configuration. Compared to MANs, campus networks are easier to set up and manage because they cover less ground and support fewer devices.
• Wi-Fi hotspot creation. Universities and businesses with campus networks might set up free Wi-Fi hotspots in areas with high volume to enable easy network access.

Campus use cases

• Campus network provisioning. Network administrators commonly set up campus networks to create networks large enough to cover a school or university.
• Enterprise network configuration. Businesses set up campus networks across a corporate campus to distribute one standardized network across buildings. This design typically costs less than other large-scale network configurations.

5. Wide area network

A wide area network ( WAN ) is a large-scale computer network. Like a MAN, a WAN is a connection of multiple LANs that belong to the same network. Unlike MANs, however, WANs aren't restricted to the confines of city limits. A WAN can, theoretically, extend to any area of the globe. For example, an organization with a corporate office in New York can connect a branch location in London within the same WAN. Users in both locations obtain access to business data, files and applications, and they can communicate with each other.

WAN benefits

• Large area coverage. A WAN can connect networks located anywhere in the world.
• Improved performance. WANs use links with dedicated bandwidth to connect LANs together. These links enhance network speeds and provide faster data transfer rates than LANs.
• Increased security. Dedicated links also reduce the risk of external attacks because traffic doesn't travel across public infrastructure, which lowers the chances for hackers to hijack a system.

WAN use cases

• Long -distance connectivity. Organizations use WANs to connect office locations separate from headquarters.
• Interconnection. Users in widely distributed locations can share files, data and communicate with each other without delay.

6. Global area network

A global area network ( GAN ) is the most expansive type of network configuration. Like other wide-ranging networks, a GAN consists of multiple interconnected networks, such as LANs and WANs. In theory, a GAN covers an unlimited geographic area, such as the entire globe. The primary difference between a WAN and a GAN is the intended scope of each network. While a WAN interconnects geographically dispersed LANs, a GAN is specifically designed to span across the entire world.

GAN benefits

• Global network coverage. GANs link networks around the world and connect devices regardless of location.
• Interconnected network. Because GANs connect networks around the world, they support organizations that need to link globally dispersed users and offices.
• Streamlined communication. GANs offer high-speed data transmission, which enables fast and efficient real-time communication and data sharing across networks worldwide.

GAN use cases

• Internet access. An estimated two-thirds of the global population uses the internet -- the world's most popular and largest GAN -- today.
• Multinational enterprise connectivity. Global corporations use GANs to connect international offices and data centers around the world.
• Connectivity for global network configurations. GANs facilitate connectivity for any network configuration that requires global infrastructure, such as cloud networks, satellite networks and international telecom networks.

7. Cloud network

A cloud network is a virtual network infrastructure composed of interconnected servers, VMs, storage, applications and other resources. Cloud service providers ( CSPs ) manage network management software and virtualized network hardware in global cloud data centers, while network administrators manage their organization's on-premises network infrastructure to enable integration with the cloud. Traditional networks, such as LANs and WANs, access the cloud network via the internet. Cloud networking differs from cloud computing as it provides the infrastructure required for cloud computing.

Cloud network benefits

• Scalability. CSPs can scale resources up or down as needed, which lets them optimize a network's workload based on computing or bandwidth requirements.
• Cost savings. When CSPs update resources based on load requirements, it prevents network overprovisioning and helps enterprises save on network costs. Additionally, virtualized network infrastructure costs less to deploy and maintain, which lowers operational costs.
• Reliability. CSPs offer several reliability measures, such as failover mechanisms and redundancy features, to ensure high availability and fault tolerance. They also provide high performance as they manage cloud networks in distributed data centers. Data centers in closer proximity to users can transfer data faster and reduce latency.

Cloud network use cases

• Remote work. Users can access data stored in a cloud network from any location, which makes cloud networks suitable for remote work.
• Managed network services. Enterprises that want to offload network management to third-party providers can deploy a cloud network, so CSPs manage the infrastructure, software and connectivity.

8. Content delivery network

A content delivery network ( CDN ) is a network of globally distributed servers that deliver dynamic multimedia content, such as interactive ads or video content, to web-based internet users. CDNs use specialized servers that cache bandwidth-heavy rich media content, which speeds up delivery time. CDN providers deploy these digitized servers globally at a network edge to create geographically distributed points of presence.

When a user requests data in a network, a proxy server forwards the data to the nearest CDN server. The proxy server encrypts the data into a smaller, more manageable file for the network to handle. It then delivers the data to an origin server, which provides the content to the user.

CDN benefits

• Fast content delivery. The main goal of a CDN is to load rich media content on websites quickly and reduce latency between requests.
• Increased security. When traffic travels through a CDN server, potential viruses attached to data reroute to the server too. A CDN service mitigates these threats so it can send uncompromised data through the network.
• Improved site performance . Websites managed by CDNs experience less latency and bandwidth limitation issues. Network downtime caused by traffic spikes is also a rare occurrence in websites with CDNs.

CDN use cases

• Rich media delivery. CDNs enable the delivery of rich, or dynamic, media. Most websites and applications incorporate some form of dynamic content, from embedded social media posts to video-streaming players. CDNs are necessary to accommodate the vast amount of complex data shared among millions of internet users each day.
• Collection of real-time analytics. When CDNs cache content from servers, they collect data about the content to optimize delivery. Enterprises can use this data in the form of real-time analytics to improve network performance.
• Software distribution. CDNs optimize software distribution for geographically dispersed users. Users located far away from servers that host software often experience longer download times, but CDNs can cache software artifacts to servers around the world. When a user downloads software through a CDN, it downloads from the closest server, which creates faster download speeds.

9. Virtual private network

A virtual private network ( VPN ) creates a private network overlay across an existing public network. VPNs use tunneling protocols that create encrypted connections between the network and client devices. Network traffic travels over the VPN service's secure, encrypted tunnels instead of a public network. This process hides a user's IP address and data from ISPs and potential hackers. The user's location appears to be wherever the VPN server exists.

VPN benefits

• Privacy and anonymity. Users can browse networks without inspection by their ISPs.
• Increased security. Users must authenticate before they gain access to a VPN. Organizations can secure company data as they prevent unauthenticated users from accessing sensitive information.
• Geo-spoofing. Users connected to VPNs appear to be in the same location as the server, whether in an office building or another country entirely. Users can retrieve company data or gain access to geo-blocked content outside of their country's borders.

VPN use cases

• Private browsing. VPNs have risen in popularity as internet users seek to browse the web without surveillance from their ISPs. An ISP can monitor a user's web activity, including sites visited and the types of content downloaded. VPNs hide this information from an ISP while still providing the user with access to the network service.
• Remote work. VPNs facilitate remote work for individuals outside of office locations. User devices with VPN client software can connect to their organization's VPN server and access their office's data center resources. They can access the same business files and resources as employees located in the building.

Additional types of networks

The networks detailed previously are those that commonly appear in an enterprise network configuration. However, network teams might also hear about the following network types as well:

• Storage area network. A storage area network, sometimes referred to as SAN , is a network that interconnects and provides network access to storage devices. Servers connected to a storage area network can access the same set of storage devices, which lets network administrators centrally manage storage, optimize efficiency and improve performance.
• System area network . A system area network, also sometimes referred to as SAN , is a network that interconnects a cluster of computers in high-performance computing ( HPC ) environments. System area networks increase bandwidth, lower latency and provide high speeds in HPC systems, all of which are essential to facilitate the complex calculations completed in these environments.
• Enterprise private network . An EPN is a network configuration privately owned and operated by an enterprise. An EPN is essentially the same as a traditional network configuration, such as a LAN or WAN, but built in-house by an organization's network team to interconnect company locations.

Which is the best type of network?

Several network types, associated topologies and connectivity methods are available, even beyond those in this overview. With so many options, network professionals might wonder which design is best for their organization. The simple answer: There isn't one. The choice largely depends on business requirements, supported applications and the purpose of the system.

Before network professionals decide which type of network to configure, they should first determine the following:

• The use cases of the network.
• The types of users and devices the network will serve.
• The location of the network.

Once they solidify those answers, then they can select which type of network and connectivity to deploy.

Editor's note: This article was updated to include additional information and improve the reader experience.

Deanna Darah is site editor for TechTarget's Networking site. She began editing and writing at TechTarget after graduating from the University of Massachusetts Lowell in 2021.

Server and networking interview questions and answers

Related Resources

• Navigating Cost and Contractual Concerns in Network Monitoring and Managed ... –LogicVein
• Wi-Fi 6 And Beyond: Your On-Ramp To The Connected Campus –SHI International
• How to build an agile enterprise network infrastructure –Console Connect
• Leeds Beckett University Modernises Campus Infrastructure with Extreme Networks –FourNet

Dig Deeper on Network infrastructure

local area network (LAN)

CNN vs. GAN: How are they different?

computer network

generative adversarial network (GAN)

Microsoft 365 Copilot, an AI assistant, offers several promising features. Find out how to configure Copilot with Teams workflows...

With its AI capabilities, Microsoft Copilot provides several enhancements to Microsoft Teams functionality, including meeting ...

Organizations have ramped up their use of communications platform as a service and APIs to expand communication channels between ...

To keep corporate and user data safe, IT must continuously ensure mobile app security. Mobile application security audits are a ...

Dell continues to cut its workforce to become 'leaner,' as it repositions for changes in the enterprise PC market that are ...

Tap to Pay makes it possible to accept customer payments from an iPhone with no additional hardware. Find out the best use cases ...

Due to rapid AI hardware advancement, companies are releasing advanced products yearly to keep up with the competition. The new ...

AMD plans to acquire AI systems designer and manufacturer ZT Systems for $5 billion. AMD CEO Lisa Su said hyperscalers want more ...

Data center modernization offers a competitive advantage to organizations, along with maximizing hyperscale infrastructure. ...

Popular pricing models for managed service providers include monitoring only, per device, per user, all-you-can-eat or ...

Global IT consultancies take a multilayered approach to GenAI training by developing in-house programs, partnering with tech ...

IT service providers are upskilling a large portion of their workforces on the emerging technology. The campaign seeks to boost ...

• Generative AI
• Office Suites
• Collaboration Software
• Productivity Software
• Augmented Reality
• Emerging Technology
• Remote Work
• Artificial Intelligence
• Operating Systems
• IT Leadership
• IT Management
• IT Operations
• Cloud Computing
• Computers and Peripherals
• Data Center
• Enterprise Applications
• Vendors and Providers
• Enterprise Buyer’s Guides
• United States
• Netherlands
• United Kingdom
• New Zealand
• Newsletters
• Foundry Careers
• Terms of Service
• Privacy Policy
• Cookie Policy
• Copyright Notice
• Member Preferences
• About AdChoices
• E-commerce Affiliate Relationships
• Your California Privacy Rights

Our Network

• Network World
• Case studies in network and systems management

These case studies exemplify how corporate IT departments are using new tools and approaches to managing their networks and servers.

Mandarin Oriental Hotel Group , a unit of Mandarin Oriental International Ltd., Hong Kong

Mission: Owner and operator of 18 luxury hotels on three continents, in locations such as Hong Kong, London, Geneva, Bermuda, Bangkok and Singapore.

Challenge: As the hotel chain grew from its Asian base to other continents, it needed a way to manage its growing network, build its infrastructure quickly and remain agile — with little capital investment, says Nick Price, director of technology (the equivalent of a CIO) at Mandarin Oriental Hotel Group in Hong Kong.

Technology: To meet those goals, the hotel group decided to outsource network management to InteQ Corp. , a managed services provider in Bedford, Mass. The vendor’s InfraService offering monitors the hotel group’s servers, applications and network devices and remotely resolves problems. (The hotel group uses an Internet-based virtual private network because of the geographic dispersion.) The Web-based interface provides a window to how the network is doing, offers real-time alerts and demonstrates the value of the service to upper management.

Payoff: “InteQ did what would have taken us two years to do,” says Geoff McClelland, vice president of technology for hotel development, based in Sydney, Australia. And it would be hard for the hotel group to maintain the high-caliber staff expertise that InteQ has, McClelland adds.

Intellinex LLC , an independent business unit launched by Ernst & Young International, Cleveland

Mission: Intellinex provides sophisticated online and multimedia educational programs for major corporate clients.

Challenge: An e-learning division that catered to 30,000 people within Ernst & Young was spun off into a company serving 175,000 people — but its IT infrastructure couldn’t handle that scale, says Mark Bockeloh, Intellinex’s chief technology officer, who is based in Las Calinas, Texas. The training programs are full of multimedia content such as streaming video, and have complex, dynamic features like bookmarking the student’s place in an online course and providing different paths through the course depending on how the student answers the questions. In addition, there are registration and scheduling chores.

Technology: Intellinex obtained San Mateo, Calif.-based MetiLinx Inc. ‘s iSystem Enterprise server optimization and management software. The suite determines the transaction mix on the servers and optimizes the traffic in real time, finding the best route and the best server to handle it, Bockeloh says. “It routes the work to the server with the greatest health, which might be Machine 10 in Miami, and then routes it to Miami,” he says.

Payoff: The result of the optimization is “46% more throughput, and no downtime,” Bockeloh says, “and that means I don’t need 46% more hardware.”

Sony Online Entertainment Inc. , San Diego

Mission: This online gaming division of Sony Corp. runs “massively multiplayer” online games — including the blockbuster EverQuest — with more than 13 million registered users.

Challenge: Sony Online has more than 1,500 servers and 100,000 simultaneous players across three continents. The company needed a network monitoring tool that was inexpensive, easy to configure, easy to manage and easy to blend with custom applications that the gaming network uses, says Adam Joffe, vice president of IT. “Our custom applications wouldn’t fit real well with any of the standard network-monitoring packages that are out there,” he explains.

Technology: Sony Online selected NetVigil from Fidelia Inc. in Princeton, N.J. NetVigil is flexible and has open application programming interfaces “so we could plug in our own custom monitoring tools,” Joffe says.

Payoff: NetVigil was a better choice than larger network management packages, Joffe says. “They would have been overkill, and we’d have paid an awful lot of money for only certain features. NetVigil gives us a broad view of all of our devices — host and network devices. Other packages do either the network well or the host well, but not necessarily both,” he says.

• Taking Control

Stories in this report:

• The Story So Far: Network and Systems Management
• Coping with bandwidth hogs
• Field Report: Network and Systems Management Tools
• Simple Network Management Protocol
• Expediting Content Delivery
• How to Thrive in the Networking Market
• Partitioning: Your Mainframe becomes a Hotel for OS ‘Guests’
• Success With Content Delivery Networks Lies in the Planning

Related content

Nerdio enables remote work across the canadian wilderness for the government of alberta, digital nomads just got huge screens and fast internet, mcdonald's serves up a master class in how not to explain a system outage, meta services begin recovering from major disruption, from our editors straight to your inbox.

Mitch Betts is an executive editor at IDG Enterprise. He was previously executive editor of CIO and Computerworld magazines.

More from this author

Buyers’ guide: prices dropping for data visualization software, it service providers get low marks for innovation, two big mistakes cios make with digital strategy, how crm buyers can negotiate the best deal, prices still climbing for enterprise mobility management software, 6 ex-cios reveal lessons learned, biggest regrets, report: security teams plagued by poorly managed identity data, idc report: streamline those ancient rfp templates, most popular authors.

• Elizabeth Montalbano

Show me more

Character.ai founder to co-lead gemini ai, says report.

5 steps to repair Microsoft Office

I've got the genAI blues

Podcast: End users need to switch their data security strategy

Podcast: Is the gold rush for AI talent slowing down?

Podcast: Google loses antitrust, and the world yawns

Why the AI money spigot is slowing down

Tech news roundup: The hackers are winning, it's time to switch security strategy

Is there still a gold rush for AI talent?

• Free Downloads
• Network Videos
• Advertise with Us
• Your Product Review

RouterFreak Reviews, Tips and Tools for Network Engineers

Software defined network: use cases from the real world, maab saleem.

Hyper-Converged storage/Converged storage

Video applications, orchestration of mobile network services, data center networks’ scalability, basically the network of any enterprise.

What do you think about this article?

Leave a reply cancel reply.

Your email address will not be published. Required fields are marked *

Notify me of followup comments via e-mail. You can also subscribe without commenting.

This site uses Akismet to reduce spam. Learn how your comment data is processed .

RouterFreak is a blog dedicated to professional network engineers. We focus on network fundamentals , product/service reviews , and career advancements .

As an Amazon Associate, I earn from qualifying purchases.

RouterFreak is supported by its audience. We may receive a small commission from the affiliate links in this post, at no extra cost to our readers.

Recommended

Popular articles

• Optimizing Network Performance with the Right Cloud Instance Types
• Mac Ransomware on the Rise: How a VPN Minimizes the Risk
• The Business Case For Investing In Website Accessibility
• Roku Keeps Restarting (Causes & Easy Fixes)
• How To Connect Vizio TV To WiFi Without Remote

• Skip to content
• Skip to search
• Skip to footer

Data Center Networking Customer Stories

Real customers. Real business transformation with Cisco Data Center Networking solutions.

• Search case studies

Contact Cisco

• Get a call from Sales

Call Sales:

• 1-800-553-6387
• US/CAN | 5am-5pm PT
• Product / Technical Support
• Training & Certification

Skanska, Rakuten, and Riot Games

Skanska: Stretching a data center to the cloud

Skanska extends its data center network to Microsoft Azure with Cisco ACI and Cisco Nexus Dashboard.

Rakuten Mobile: Advancing, scaling to support demands

Rakuten Mobile builds world's first Cloud-native, software-defined 5G network with Cisco ACI, Cisco Nexus Series Switches, and Cisco Crosswork.

Riot Games: Breaking traditional broadcast molds

Riot Games revolutionizes sports, media, and entertainment with Cisco Data Center Networking and Cisco Compute technologies.

Executive perspectives

Daimler AG takes the first step on their journey toward a zero-trust, software-defined, globally distributed IT environment with Cisco Nexus Series Switches and NX-OS.

Turkcell relies on Cisco ACI, Cisco Nexus Series Switches, and Cisco NSO to put zero-touch, zero-trust network operations to the test.

Great Southern Bank and University of Melbourne

Network automation is being realized in different ways, for different needs, and with different Cisco Data Center Networking solutions.

All Case Studies

Sorry, no results matched your search criteria(s). Please try again.

11 Types of Networks: Understanding the Differences

Last Updated: August 19, 2024

As the number of connected personal and IoT devices skyrockets across the world , the demand for high-speed, high-performance networks continues to climb and transform how people and businesses connect. But with so many different types of networks, how do you choose the right one for optimal performance and security?

Technological growth and shifting demands have triggered the development of these different types of networks. It’s likely that with future technological advancement, even more network topology types will continue to emerge. Reducing blind spots on your network and introducing network monitoring tools have never been more important.

We’ve put together a list of common network types and provided some guidance about choosing one. After reading this article you’ll:

• Learn about the most common network topologies and terminologies
• Understand how to choose between different types of networks and their connections

So whether you’re network veteran or managing a new network , you’ll be sure to get something from this article. Let’s jump right in!

What are the different types of networks?

Networks come in various forms, each tailored to specific needs and scales, from local area networks (LANs) that connect devices within a single location to wide area networks (WANs) that span across large geographical areas. Understanding these different types of networks is crucial for designing and managing efficient, secure, and scalable IT environments.

Local Area Network (LAN)

When your laptop or mobile phone is connected into your home or office network, you’re using what’s known as the local area network (LAN).

A LAN is a proprietary computer network that enables designated users to have exclusive access to the same system connection at a common location, always within an area of less than a mile and most often within the same building. By doing so they’re able to share devices, share resources such as printers, and exchange information as if they were all working from the same system. Resource sharing is possible with a network-aware operating system.

Originally used in universities and research labs, today LANs are in use everywhere, including in the home and business. With the use of coaxial cables, optical fiber cables, or twisted wire pair, different types of network topologies, such as bus, star, and branching tree are used to fulfill specific goals. Information sharing and communication between devices over different topologies is possible with Wi-Fi or TCP/IP Ethernet.

When to use a LAN:

• If you have many users needing to connect at a single location.
• When devices and resources need to communicate directly with one another within a small geographic area.

When not to use a LAN:

• Connecting users across vast distances, such as in separate cities.
• If you don’t have control over the assets connecting to your network, you may want to be cautious on setting up a LAN for them to communicate.

Personal Area Network (PAN)

A personal area network (PAN) is a short-range network topology designed for peripheral devices (usually 30ft) used by an individual. The purpose of these types of networks is to transmit data between devices without being necessarily connected to the internet.

PANs can also be connected to LAN and higher level network types where one device acts as a gateway. An everyday example of PAN is a Bluetooth keyboard that’s connected to a smart TV, where the interface allows you to surf the internet, view recorded programs, and configure personal preferences.

Personal area networks can either be wireless or wired. Wireless PANs are called WPANs and use close-range connectivity protocols such as Wi-Fi, ZigBee, infrared, and Bluetooth for data-centric applications. In the case of Bluetooth, network configurations can be piconet—involving a master and one or more slaves—or scatternets, which are interrelated piconets.

Wired PANs, on the other hand, use universal serial bus (USB) and ThunderBolt. Like other network types, each connectivity protocol within a PAN is typically not directly compatible with other protocols.

When to use a PAN:

• You’re looking to connect accessories or peripherals to laptops, cell phones, etc.
• The devices generally don’t need to connect directly to the internet.

When not to use a PAN:

• You’re looking to share resources between different users.
• The distance between the devices is more than a few feet.

Wireless Local Area Network (WLAN)

Wireless technologies have been a major breakthrough in commercial and personal connectivity, opening up numerous possibilities ranging from mobile wireless, fixed wireless, portable wireless, or IR wireless. Connected devices on these configurations communicate over what’s known as a wireless local area network (WLAN).

WLANs use high-frequency signals, lasers, and infrared beams to enable devices (also known as clients) to communicate with each other without the need of electrical conductors (wires) to transmit data. This type of flexible data communication makes it easy for users to move around a coverage area without the need of cables to maintain network connectivity.

WLAN provides a high data transfer rate and most often works in the 2.4 GHz band or 5 GHz band. Some examples of devices that operate in the 2.4 GHz band over a WLAN include Bluetooth devices, cordless telephones, Wi-Fi radios, and garage door openers. If you’re looking to sacrifice some signal strength for better speed in a WLAN, laptops can be connected to the 5 GHz band.

When to use a WLAN:

• Mobility of devices while connected to the network is important.
• Your devices don’t support a type of wired network connection.
• You need to connect devices not physically close to existing network infrastructure.
• There are more devices you need to connect than the number of ports on your router or switch and you can’t add an additional switch or router.

When not to use a WLAN:

• Consistent unwavering performance is important.
• Security is the only priority.
• Data transfer rates exceed those available through wireless technologies.

Wide Area Network (WAN)

Private lines, virtual private networks (VPNs), multiprotocol label switching (MPLS), wireless networks, cellular networks, and the internet allow LANs and other types of networks in different geographical regions to communicate and transmit data. This type of computer networking is known as a wide area network (WAN), a telecommunication network that’s not limited to any particular geography, providing access to various forms of media through a designated provider. WANs can be basic or hybrid with point-to-point or packet-switched networks over shared circuits. In the case of a hybrid WAN and SD-WAN , different connection types are used that can range from virtual private networks (VPNs) and multiprotocol label switching (MPLS). Communication channels within a WAN often feature a wide range of different technologies, ranging from routers, FSO links, and I/O interfaces to fiber optics.

While they often go unnoticed, WANs are embedded everywhere in modern life, connecting cities, countries, and even space. From providing remote access to a corporation’s head office, allowing students to communicate with other students in different continents, to teleconferencing in real time, these and many other examples show how far-flung WANs have become.

When to use a WAN:

• You have devices spread over a wide geography that need to communicate directly with each other.

When not to use a WAN:

• Cost is the only priority, as some WAN technologies such as leased lines can become cost prohibitive.
• Consistent performance is a primary requirement, as some WAN technologies can vary in performance.

Metropolitan Area Network (MAN)

In today’s world of computer networking, efficiency and speed are top priorities. Some technologies manage to deliver both, while others have only one feature or none at all. A metropolitan area network (MAN) ticks both boxes by using technologies such as fiber optics, dense wavelength division multiplexing, and optical packet switching.

Typical layout of a metropolitan area network (MAN) Source: Science Direct Often referred to as medium-sized networks, MANs covers an area larger than a LAN, but smaller than a WAN. They consist of different LANs interconnected with point-to-point high-capacity backbone technology and can span several buildings or a metropolis.

Through shared regional resources, MANs can take the form of cable TV network, or even telephone networks that provide high-speed DSL lines.

When to use a MAN:

• You have devices spread over a regional geography that need to communicate directly with each other.
• You have the ability to provide connectivity, wired or wireless, between each location.

When not to use a MAN:

• Cost is a primary requirement, as less costly solutions to connect sites may exist.
• Devices to be connected are spread out over a larger geographic area.

Campus Area Network (CAN)

With a campus area network (CAN), universities, colleges, and corporate campuses connect different LANs from various departments sharing a common area. This transforms otherwise scattered networks into a collective network that provides access to information at breathtaking speeds while ensuring the necessary authentication to prevent privacy loopholes.

CANs are similar to LANs in operational approach, but differ in size to these types of networks. Users who access a CAN with different devices often do so with Wi-Fi, hotspots, and Ethernet technology.

When to use a CAN:

• You have devices spread over campus that need to communicate directly with each other.
• You have the ability to provide connectivity, wired or wireless, between each building.

When not to use a CAN:

• Devices to be connected are spread out over a larger geographic area, not isolated to just the campus.

Virtual Private Network (VPN)

With cyberattacks lurking in every click and the risk of having sensitive information mined, intercepted, or even stolen, a virtual private network (VPN) offers users an encrypted connection that effectively hides data packets while using the internet.

This is achieved with a VPN tunnel that’s created between two communicating devices, encapsulating and encrypting the data transferred between the two devices. Typically a VPN is used when the two devices are connected over a public network, such as the internet. The extra protection offered by the VPN tunnel prevents sensitive information such as IP addresses, surfing history, communication with a corporate office, or even travelling plans from being exposed online.

The level of security surrounding a data packet depends on the type of VPN tunnel used. Typical VPN tunnels include point-to-point tunneling protocol (PPTP), Secure Socket Tunneling Protocol (SSTP), L2TP/IPsec, and OpenVPN.

Layer 2 Tunneling Protocol (L2TP) which uses the Internet Protocol Security (IPsec) protection typically does so with AES-256 bit encryption , an advanced encryption standard considered to be the strongest available for all types of network connections.

There are different types of VPNs , which can generally be split into two categories: remote access VPN and site-to-site VPN. With remote access VPNs, users securely connect their devices to the corporate office. With site-so-site VPN, connection is done from a corporate office to branch.

When to use a VPN:

• You need to facilitate secure communications between two locations but don’t have any direct connectivity between the two locations.
• You need to provide remote access to resources at a central location to remote users.

When not to use a VPN:

• Speed is your only concern. The encryption process adds slight overhead to communications.

Enterprise Private Network (EPN)

Bandwidth-intensive applications use a huge chunk of company network resources, slowing down data transfer and leading to bottlenecks in business operations. An enterprise private network (EPN) is a custom-design network, built and operated by a business to share company resources. It connects a company’s offices across different geographic regions and is optimized to ensure that bandwidth-centric applications run smoothly without burdening the network.

With an EPN, companies can choose to have a purpose-built network that’s fully private or a hybrid integrated with a network Communications Service Provider (CSP). EPNs are optimized with tunneling protocols, such as Layer 2 Tunneling Protocol (L2TP) and Internet Protocol Security (IPsec) to ensure privacy across all network operations. Branches are connected with MPLS technology.

When to use an EPN:

• You need to provide users spread across various locations consistent access to resources spread across various locations.
• There are requirements to scale and grow the network over time.

When not to use an EPN:

• Your network team is resource-constrained. EPNs require additional time to set up, update, and maintain.
• Redundant links between locations aren’t possible, as this will introduce additional failure points to the network.

Storage Area Network (SAN)

Network storage is synonymous with business continuity in an increasingly competitive world. Businesses that want to stay ahead, need to find ways to optimize data access and data storage, and ensure that important backups are done on a regular basis. One way to achieve these aims and more is by using a storage area network (SAN).

A storage area network (SAN), or network behind the servers, is a special purpose high-speed computer network that provides any-to-any access to storage. The main purpose of a SAN is to transfer data between different storage devices and between the computer network and storage devices.

Block-level I/O services are characteristic of most SANs. Different components used in a SAN may include fiber channel technologies such as fiber channel host bus adapter (HBA) cards and fiber channel switches and other technologies such as hosts, switches, and disk arrays.

When to use a SAN:

• There are multiple devices that need to share the resources available on one or more storage devices.
• You require centralized storage for data across all resources.

When not to use a SAN:

• You’re budget constrained. There may be significant upfront investment to establish a SAN.
• You’re resource constrained. SANs may require additional maintenance and upkeep compared to local storage.

System Area Network (also referred to as SAN)

A system area network (SAN) is a type of network configuration designed to facilitate communication between nodes in a cluster. It’s designed to provide high bandwidth and ensure low latency by avoiding multiple copies of data and providing direct network access to users in a high performance computing environment.

As a result, SANs ensure high-speed switched environments that facilitate network communication between devices. Interconnections with multiprocessing systems (processor-to-processor) and storage area systems (SANs) are also possible with a SAN.

As SANs are designed to be used in parallel computing environments, typical examples include use in scientific applications, database server clusters, and file server clusters.

When to use a system area network:

• You require the data to be transferred with low error rates, high bandwidth, and low latency.
• You’re not cost-constrained.

When not to use a system area network:

• The requirements for your network don’t include a sizable investment that may be required to set up a system area network.

Passive Optical Local Area Network (POLAN)

Copper-based connectivity was once widely accepted in local area network (LAN) installations. Today this scenario is changing as passive optical local area networks (POLAN) edge out older installations.

POLAN is a significant upgrade from copper cables, replacing them with fiber-optic telecommunications technology that uses optical splitters to split and combine upstream and downstream signals that are eventually sent on a strand of single-mode fiber.

The fiber bandwidth is divided among different access points and the use of wavelength division multiplexing (WDM) enables bi-directional communication that reduces the number of copper cables.

Examples where POLAN are used include campus buildings where departments share a common network, hospitals that have a shared network with on-site pharmacy and patient needs, and any other LAN network types.

When to use a POLAN:

• You need a lower cost solution for connecting remote locations and have the ability to install optical fiber between the locations.

When not to use a POLAN:

• You require a well defined standard for your network. There are currently no accepted worldwide standards that define POLAN installations and this could lead to compatibility issues in the future.
• You’re budget constrained.

How to choose the right types of networks for your organization

Network types clearly differ in range and applicability so choosing a network for your business means aligning with your business goals, the long-term outlook, and the physical realities of the devices that you’re looking to connect.

There are four important factors to consider when looking at different types of networks.

• Purpose: Networks are purpose-specific and configured to fulfill performance goals. For instance, a system area network (SAN) is better suited for environments with high-performance computing requirements. Combining a SAN with, for example, a CAN on a corporate campus requires a thorough study to understand how these different types of networking topologies can complement each other to avoid high capital costs and idle resources.
• Cost: Upfront and long-term costs should be weighed against performance metrics. Is it better to invest in a WAN or would an EPN be better? Maintenance costs are often overlooked during the investment stages and cause great surprises eventually. Knowing how costs will evolve helps build the business case of choosing a particular network over the other.
• Availability: Network availability involves more than just being connected. Spare parts, upgrades, and software change are all part of availability. Is a new technology being used? How will spare parts and updates be handled?
• Scalability: As more devices connect to a network, changes will need to be made to reinforce security, provide more bandwidth, ensure better speeds, and consider growth. To consider scalability means planning for the future.

No matter what type of network you choose to implement, Auvik can help you manage it efficiently and effectively.

Everything you need to know about network topology

From network layers and components to segmentation to step-by-step instructions for drawing every layer.

What do you think?

Share your thoughts and read what others have to say

Very Interesting and informative

Very interesting and has an excellent presentation. Pretrysehuk has the mind of sharing his views to others.

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Share your thoughts here *

Enter your name *

Enter your email address *

Save my name, email, and website in this browser for the next time I comment.

How to Create a Network Assessment Report with This Template

Network diagrams 101: definitions, examples & how-to’s, what is network visualization, what’s your shadow it risk factor.

• The impact shadow IT has on an organization
• How to evaluate tools
• Tips on security
• A quiz to help you determine the severity of shadow IT in your org
• Solutions to solve these problems

Computer Networks

Computer Networking: Case Study Questions

• Categories Computer Networks , Networking , Computer Networks

This post contains case study questions on Computer Networking.

Case study 1:.

Web server is a special computer system running on HTTP through web pages. The web page is a medium to carry data from one computer system to another. The working of the webserver starts from the client or user. The client sends their request through the web browser to the webserver. Web server takes this request, processes it and then sends back processed data to the client. The server gathers all of our web page information and sends it to the user, which we see on our computer system in the form of a web page. When the client sends a request for processing to the web server, a domain name and IP address are important to the webserver. The domain name and IP address are used to identify the user on a large network.

• IP addresses
• Computer systems
• Webpages of a site
• A medium to carry data from one computer to another
• Home address
• Domain name
• Both b and c
• Hypertext Transfer Protocol
• Hypertext Transfer Procedure
• Hyperlink Transfer Protocol
• Hyperlink Transfer Procedure
• Domain name system
• Routing information protocol
• Network time protocol
• None of the above
• Domain Name Security
• Domain Number System
• Document Name System
• Domain Name System

Case Study 2:

In mid 80’s another federal agency, the NSF created a new high capacity network called NSFnet, which was more capable than ARPANET. The only drawback of NSFnet was that it allowed only academic research on its network and not any kind of private business on it. Now, several private organisations and people started working to build their own networks, named private networks, which were later (in 1990’s) connected with ARPANET and NSFnet to form the Internet. The Internet really became popular in 1990’s after the development of World Wide Web.

• National Senior Foundation Network
• National Science Framework Network
• National Science Foundation Network
• National Science Formation Network
• Advanced Research Premium Agency NETwork
• Advanced Research Projects Agency NETwork
• Advanced Review Projects Agency NETwork
• Advanced Research Protection Agency NETwork
• A single network
• A vast collection of different networks
• Interconnection of local area networks
• Interconnection of wide area networks
• Internet architecture board
• Internet society
• Internet service provider
• Different computer
• Leased line
• Digital subscriber line
• Digital signal line
• Digital leased line

Case Study 3:

TCP/IP, or the Transmission Control Protocol/Internet Protocol, is a suite of communication protocols used to interconnect network devices on the internet. TCP/IP can also be used as a communications protocol in a private computer network (an intranet or an extranet).

TCP defines how applications can create channels of communication across a network. It also manages how a message is assembled into smaller packets before they are then transmitted over the internet and reassembled in the right order at the destination address.

IP defines how to address and route each packet to make sure it reaches the right destination. Each gateway computer on the network checks this IP address to determine where to forward the message. TCP/IP uses the client-server model of communication in which a user or machine (a client) is provided a service (like sending a webpage) by another computer (a server) in the network. Collectively, the TCP/IP suite of protocols is classified as stateless, which means each client request is considered new because it is unrelated to previous requests. Being stateless frees up network paths so they can be used continuously.

• All of the above
• Remote procedure call
• Internet relay chat
• Resource reservation protocol
• Local procedure call
• communication between computers on a network
• metropolitan communication
• sesson layer
• transport layer
• network layer
• data link layer

Case Study 4:

A blog is a publication of personal views, thoughts, and experience on web links. It is a kind of personal diary note about an individual. The contents published on a blog are organized in a reverse manner, it means recent posts appear first and the older posts are further downwards.

Blogger – a person who posts a blog in the form of text, audio, video, weblinks, etc is known as a blogger. Bloggers have followers who follow them to get instant messages post by the blogger.

In most cases, celebrities, business tycoons, famous politicians, social workers, speakers, etc are the successful blogger because people follow them to know about their success stories and ideas.

• social networking
• social networking sites
• e-commerce websites
• search engines
• entertainment sites
• social network
• entertainment
• search engine
• none of these

Which of the following is an example of micro-blogging?

Which of the following is not used as blogging platform?

• discussion boards

Case Study 5:

An email is a service of sending or receiving emails or messages in the form of text, audio, video, etc over the internet. Various service providers are providing email services to users. The most popular service providers in India are Gmail, Yahoo, Hotmail, Rediff, etc.

An email address for an email account is a unique ID. This email ID is used to send and receive mails over the Internet. Each email address has two primary components: username and domain name. The username comes first, followed by the @) symbol and then the domain name.

• none of the above

Which of the following is the correct format of email address?

• name@website@info
• [email protected]
• www.nameofwebsite.com
• name.website.com
• multipurpose internet mail extensions
• multipurpose internet mail email
• multipurpose internet mail end
• multipurpose internet mail extra
• mail server
• user agents

NVT stands for

• network virtual transmission
• network virtual test
• network virtual terminal
• network virtual tell

Case study 6:

In 1989, Tim Berners Lee, a researcher, proposed the idea of World Wide Web). Tim Berners Lee and his team are credited with inventing Hyper Text Transfer Protocol (HTTP), HTML and the technology for a web server and a web browser. Using hyperlinks embedded in hypertext the web developers were able to connect web pages. They could design attractive webpages containing text, sound and graphics. This change witnessed a massive expansion of the Internet in the 1990s.

• A program that can display a webpage
• A program used to view HTML documents
• It enables a user to access the resources of internet
• a) is same every time whenever it displays
• b) generates on demand by a program or a request from browser
• c) both is same every time whenever it displays and generates on demand by a program or a request from browser
• d) is different always in a predefined order
• a) unique reference label
• b) uniform reference label
• c) uniform resource locator
• d) unique resource locator
• a) asynchronous javascript and xml
• b) advanced JSP and xml
• c) asynchronous JSP and xml
• d) advanced javascript and xml
• a) convention for representing and interacting with objects in html documents
• b) application programming interface
• c) hierarchy of objects in ASP.NET
• d) scripting language
• a) VBScript
• a) sent from a website and stored in user’s web browser while a user is browsing a website
• b) sent from user and stored in the server while a user is browsing a website
• c) sent from root server to all servers
• d) sent from the root server to other root servers

Case study 7:

E-business, commonly known as electronic or online business is a business where an online transaction takes place. In this transaction process, the buyer and the seller do not engage personally, but the sale happens through the internet. In 1996, Intel’s marketing and internet team coined the term “E-business

E-Commerce stands for electronic commerce and is a process through which an individual can buy, sell, deal, order and pay for the products and services over the internet. In this kind of transaction, the seller does not have to face the buyer to communicate. Few examples of e-commerce are online shopping, online ticket booking, online banking, social networking, etc.

• doing business
• sale of goods
• doing business electronically
• all of the above

Which of the following is not a major type of e-commerce?

• consolidation
• preservation
• reinvention

The primary source of financing during the early years of e-commerce was _______

• large retail films
• venture capital funds
• initial public offerings
• small products
• digital products
• specialty products
• fresh products
• value proposition
• competitive advantage
• market strategy
• universal standards

Case study 8:

Due to the rapid rise of the internet and digitization, Governments all over the world are initiating steps to involve IT in all governmental processes. This is the concept of e-government. This is to ensure that the Govt. administration becomes a swifter and more transparent process. It also helps saves huge costs.

E-Group is a feature provided by many social network services which helps you create, post, comment to and read from their “own interest” and “niche-specific forums”, often over a virtual network. “Groups” create a smaller network within a larger network and the users of the social network services can create, join, leave and report groups accordingly. “Groups” are maintained by “owners, moderators, or managers”, who can edit posts to “discussion threads” and “regulate member behavior” within the group.

• can be defined as the “application of e-commerce technologies to government and public services .”
• is the same as internet governance
• can be defined as “increasing the participation in internet use by socially excluded groups”
• Individuals in society
• computer networks
• Tax Deduction Account Number
• Tax Deduction and Collection Account Number
• Taxable Account Number
• Tax Account Number
• who conduct seminars
• who get together on weekends
• who have regular video conferences
• having the ability to access and contribute to forum topics

Case study 9:

Coursera has partnered with museums, universities, and other institutions to offer students free classes on an astounding variety of topics. Students can browse the list of available topics or simply answer the question “What would you like to learn about?”, then when they answer that question they are led to a list of available courses on that topic. Students who are nervous about getting in over their heads can relax.

• Mobile Online Open Courses
• Massive Online Open Courses
• Mobile Open Online Courses
• Massive Open Online Courses
• Blended learning
• Distance learning
• Synchronous learning
• Asynchronous learning
• Induction to the company for new employees
• Microsoft excel training
• Team-building exercise
• Building your assertiveness skills at work
• Learners using technology in a classroom environment lead by a tutor
• Training course done by youtube tutorials
• An online learning environment accessed through the internet (i.e. webinars)
• An online learning course
• MasterClass
• SimplyCoding

Case study 10:

Search Engines allow us to filter the tons of information available on the internet and get the most accurate results. And while most people don’t pay too much attention to search engines, they immensely contribute to the accuracy of results and the experience you enjoy while scouring through the internet.

Besides being the most popular search engine covering over 90% of the worldwide market, Google boasts outstanding features that make it the best search engine in the market. It boasts cutting-edge algorithms, easy-to-use interface, and personalized user experience. The platform is renowned for continually  updating its search engine  results and features to give users the best experience.

• Software systems that are designed to search for information on the world wide web 
• Used to search documents
• Used to search videos
• Single word
• Search engine pages
• Search engine result pages
• Web crawler
• Web indexer
• Web organizer
• Web manager
• Ink directory
• Search optimizer
• Generating cached files
• Affecting the visibility
• Getting meta tags
• All of these

Previous post

Insertion Sort

Mobile telecommunication technologies, you may also like.

Switching Techniques used in computer network

Communication Channels

Leave A Reply Cancel reply

You must be logged in to post a comment.

Login with your site account

Remember Me

Not a member yet? Register now

Register a new account

I accept the Terms of Service

Are you a member? Login now

In the tech world and beyond, new 5G applications are being discovered every day. From driverless cars to smarter cities, farms, and even shopping experiences, the latest standard in wireless networks is poised to transform the way we interact with information, devices and each other. What better time to take a closer look at how humans are putting 5G to use to transform their world.

What is 5G?

5G (fifth-generation mobile technology)  is the newest standard for cellular networks. Like its predecessors, 3G, 4G and 4G LTE, 5G technology uses radio waves for data transmission. However, due to significant improvements in latency, throughput and bandwidth, 5G is capable of faster download and upload speeds than previous networks.

Since its release in 2019, 5G broadband technology has been hailed as a breakthrough technology with significant implications for both consumers and businesses. Primarily, this is due to its ability to handle large volumes of data that is generated by complex devices that use its networks.

As mobile technology has expanded over the years, the number of data users generate every day has increased exponentially. Currently, other transformational technologies such as  artificial intelligence (AI) , the  Internet of Things (IoT) and  machine learning (ML)  require faster speeds to function than 3G and 4G networks offer. Enter 5G, with its lightning-fast data transfer capabilities that allow newer technologies to function in the way they were designed to.

Here are some of the biggest differences between 5G and previous wireless networks.

• Physical footprint : The transmitters that are used in 5G technology are smaller than in predecessors’ networks, allowing for discrete placement in out-of-the-way places. Furthermore, “cells”—geographical areas that all wireless networks require for connectivity—in 5G networks are smaller and require less power to run than in previous generations.
• Error rates : 5G’s adaptive Modulation and Coding Scheme (MCS), a schematic that wifi devices use to transmit data, is more powerful than ones in 3G and 4G networks. This makes 5G’s Block Error Rate (BER)—a metric of error frequency—much lower. 
• Bandwidth : By using a broader spectrum of radio frequencies than previous wireless networks, 5G networks can transmit on a wider range of bandwidths. This increases the number of devices that they can support at any given time.
• Lower latency : 5G’s low  latency , a measurement of the time it takes data to travel from one location to another, is a significant upgrade over previous generations. This means that routine activities like downloading a file or working in the cloud is going to be faster with a 5G connection than a connection on a different network.

Like all wireless networks, 5G networks are separated into geographical areas that are known as cells. Within each cell, wireless devices—such as smartphones, PCs, and IoT devices—connect to the internet via radio waves that are transmitted between an antenna and a base station. The technology that underpins 5G is essentially the same as in 3G and 4G networks. But due to its lower latency, 5G networks are capable of delivering faster download speeds—in some cases as high as 10 gigabits per second (Gbps).

As more and more devices are built for 5G speeds, demand for 5G connectivity is growing. Today, many popular Internet Service Providers (ISPs), such as Verizon, Google and AT&T, offer 5G networks to homes and businesses. According to Statista,  more than 200 million homes  (link resides outside ibm.com) and businesses have already purchased it with that number expected to at least double by 2028.

Let’s take a look at three areas of technological improvement that have made 5G so unique.

New telecom specifications

The 5G NR (New Radio) standard for cellular networks defines a new radio access technology (RAT) specification for all 5G mobile networks. The 5G rollout began in 2018 with a global initiative known as the 3rd Generation Partnership Project (3FPP). The initiative defined a new set of standards to steer the design of devices and applications for use on 5G networks.

The initiative was a success, and 5G networks grew swiftly in the ensuing years. Today, 45% of networks worldwide are 5G compatible, with that number forecasted to rise to 85% by the end of the decade according to  a recent report by Ericsson  (link resides outside ibm.com).

Independent virtual networks (network slicing)

On 5G networks, network operators can offer multiple independent virtual networks (in addition to public ones) on the same infrastructure. Unlike previous wireless networks, this new capability allows users to do more things remotely with greater security than ever before. For example, on a 5G network, enterprises can create use cases or business models and assign them their own independent virtual network. This dramatically improves the user experience for their employees by adding greater customizability and security.

Private networks

In addition to network slicing, creating a 5G private network can also enhance personalization and security features over those available on previous generations of wireless networks. Global businesses seeking more control and mobility for their employees increasingly turn to private 5G network architectures rather than public networks they’ve used in the past.

Now that we better understand how 5G technology works, let’s take a closer look at some of the exciting applications it’s enabling.

Autonomous vehicles

From taxi cabs to drones and beyond, 5G technology underpins most of the next-generation capabilities in autonomous vehicles. Until the 5G cellular standard came along, fully autonomous vehicles were a bit of a pipe dream due to the data transmission limitations of 3G and 4G technology. Now, 5G’s lightning-fast connection speeds have made transport systems for cars, trains and more, faster than previous generations, transforming the way systems and devices connect, communicate and collaborate.

Smart factories

5G, along with AI and ML, is poised to help factories become not only smarter but more automated, efficient, and resilient. Today, many mundane but necessary tasks that are associated with equipment repair and optimization are being turned over to machines thanks to 5G connectivity paired with AI and ML capabilities. This is one area where 5G is expected to be highly disruptive, impacting everything from fuel economy to the design of equipment lifecycles and how goods arrive at our homes.

For example, on a busy factory floor, drones and cameras that are connected to smart devices that use the IoT can help locate and transport something more efficiently than in the past and prevent theft. Not only is this better for the environment and consumers, but it also frees up employees to dedicate their time and energy to tasks that are more suited to their skill sets.

Smart cities

The idea of a hyper-connected urban environment that uses 5G network speeds to spur innovation in areas like law enforcement, waste disposal and disaster mitigation is fast becoming a reality. Some cities already use 5G-enabled sensors to track traffic patterns in real time and adjust signals, helping guide the flow of traffic, minimize congestion, and improve air quality.

In another example, 5G power grids monitor supply and demand across heavily populated areas and deploy AI and ML applications to “learn” what times energy is in high or low demand. This process has been shown to significantly impact energy conservation and waste, potentially reducing carbon emissions and helping cities reach sustainability goals.

Smart healthcare

Hospitals, doctors, and the healthcare industry as a whole already benefit from the speed and reliability of 5G networks every day. One example is the area of remote surgery that uses robotics and a high-definition live stream that is connected to the internet via a 5G network. Another is the field of mobile health, where 5G gives medical workers in the field quick access to patient data and medical history. This enables them to make smarter decisions, faster, and potentially save lives.

Lastly, as we saw during the pandemic, contact tracing and the mapping of outbreaks are critical to keeping populations safe. 5G’s ability to deliver of volumes of data swiftly and securely allows experts to make more informed decisions that have ramifications for everyone.

5G paired with new technological capabilities won’t just result in the automation of employee tasks, it will dramatically improve them and the overall  employee experience . Take virtual reality (VR) and augmented reality (AR), for example. VR (digital environments that shut out the real world) and AR (digital content that augments the real world) are already used by stockroom employees, transportation drivers and many others. These employees rely on wearables that are connected to a 5G network capable of high-speed data transfer rates that improve several key capabilities, including the following:

• Live views : 5G connectivity provides live, real-time views of equipment, events, and even people. One way in which this feature is being used in professional sports is to allow broadcasters to remotely call a sporting event from outside the stadium where the event is taking place.
• Digital overlays : IoT applications in a warehouse or industrial setting allow workers that are equipped with smart glasses (or even just a smartphone) to obtain real-time insights from an application. This includes repair instructions or the name and location of a spare part.
• Drone inspections : Right now, one of the leading causes of employee injury is inspection of equipment or project sites in remote and potentially dangerous areas. Drones, which are connected via 5G networks, can safely monitor equipment and project sites and even take readings from hard-to-reach gauges.

Edge computing , a computing framework that allows computations to be done closer to data sources, is fast becoming the standard for enterprises. According to  this Gartner white paper  (link resides outside ibm.com), by 2025, 75% of enterprise data will be processed at the edge (compared to only 10% today). This shift saves businesses time and money and enables better control over large volumes of data. It would be impossible without the new speed standards that are generated by 5G technology. 

Ultra-reliable edge computing and 5G enable the enterprise to achieve faster transmission speeds, increased control and greater security over massive volumes of data. Together, these twin technologies will help reduce latency while increasing speed, reliability and bandwidth, resulting in faster, more comprehensive data analysis and insights for businesses everywhere.

5G solutions with IBM Cloud Satellite  

5G presents significant opportunities for the enterprise, but first, you need a platform that can handle its speed. IBM Cloud Satellite® lets you deploy and run apps consistently across on-premises, edge computing and public cloud environments on a 5G network. And it’s all enabled by secure and auditable communications within the IBM Cloud®.

Computer Networking: Case Study Analysis

• September 2022
• Affiliation: University of Plymouth

• University of Plymouth

Abstract and Figures

Discover the world's research

• 25+ million members
• 160+ million publication pages
• 2.3+ billion citations
• Chennai Cisco
• Computernetworkingnotes
• Recruit researchers
• Join for free
• Login Email Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google Welcome back! Please log in. Email · Hint Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google No account? Sign up

arXiv's Accessibility Forum starts next month!

Help | Advanced Search

Computer Science > Software Engineering

Title: modeling network architecture: a cloud case study.

Abstract: The Internet s ability to support a wide range of services depends on the network architecture and theoretical and practical innovations necessary for future networks. Network architecture in this context refers to the structure of a computer network system as well as interactions among its physical components, their configuration, and communication protocols. Various descriptions of architecture have been developed over the years with an unusually large number of superficial icons and symbols. This situation has created a need for more coherent systematic representations of network architecture. This paper is intended to refine the design, analysis, and documentation of network architecture by adopting a conceptual model called a thinging (abstract) machine (TM), which views all components of a network in terms of a single notion: the flow of things in a TM. Since cloud computing has become increasingly popular in the last few years as a model for a shared pool of networks, servers, storage, and applications, we apply the TM to model a real case study of cloud networks. The resultant model introduces an integrated representation of computer networks.

Submission history

Access paper:.

• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

DBLP - CS Bibliography

Bibtex formatted citation.

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

8 case studies and real world examples of how Big Data has helped keep on top of competition

Fast, data-informed decision-making can drive business success. Managing high customer expectations, navigating marketing challenges, and global competition – many organizations look to data analytics and business intelligence for a competitive advantage.

Using data to serve up personalized ads based on browsing history, providing contextual KPI data access for all employees and centralizing data from across the business into one digital ecosystem so processes can be more thoroughly reviewed are all examples of business intelligence.

Organizations invest in data science because it promises to bring competitive advantages.

Data is transforming into an actionable asset, and new tools are using that reality to move the needle with ML. As a result, organizations are on the brink of mobilizing data to not only predict the future but also to increase the likelihood of certain outcomes through prescriptive analytics.

Here are some case studies that show some ways BI is making a difference for companies around the world:

1) Starbucks:

With 90 million transactions a week in 25,000 stores worldwide the coffee giant is in many ways on the cutting edge of using big data and artificial intelligence to help direct marketing, sales and business decisions

Through its popular loyalty card program and mobile application, Starbucks owns individual purchase data from millions of customers. Using this information and BI tools, the company predicts purchases and sends individual offers of what customers will likely prefer via their app and email. This system draws existing customers into its stores more frequently and increases sales volumes.

The same intel that helps Starbucks suggest new products to try also helps the company send personalized offers and discounts that go far beyond a special birthday discount. Additionally, a customized email goes out to any customer who hasn’t visited a Starbucks recently with enticing offers—built from that individual’s purchase history—to re-engage them.

2) Netflix:

The online entertainment company’s 148 million subscribers give it a massive BI advantage.

Netflix has digitized its interactions with its 151 million subscribers. It collects data from each of its users and with the help of data analytics understands the behavior of subscribers and their watching patterns. It then leverages that information to recommend movies and TV shows customized as per the subscriber’s choice and preferences.

As per Netflix, around 80% of the viewer’s activity is triggered by personalized algorithmic recommendations. Where Netflix gains an edge over its peers is that by collecting different data points, it creates detailed profiles of its subscribers which helps them engage with them better.

The recommendation system of Netflix contributes to more than 80% of the content streamed by its subscribers which has helped Netflix earn a whopping one billion via customer retention. Due to this reason, Netflix doesn’t have to invest too much on advertising and marketing their shows. They precisely know an estimate of the people who would be interested in watching a show.

3) Coca-Cola:

Coca Cola is the world’s largest beverage company, with over 500 soft drink brands sold in more than 200 countries. Given the size of its operations, Coca Cola generates a substantial amount of data across its value chain – including sourcing, production, distribution, sales and customer feedback which they can leverage to drive successful business decisions.

Coca Cola has been investing extensively in research and development, especially in AI, to better leverage the mountain of data it collects from customers all around the world. This initiative has helped them better understand consumer trends in terms of price, flavors, packaging, and consumer’ preference for healthier options in certain regions.

With 35 million Twitter followers and a whopping 105 million Facebook fans, Coca-Cola benefits from its social media data. Using AI-powered image-recognition technology, they can track when photographs of its drinks are posted online. This data, paired with the power of BI, gives the company important insights into who is drinking their beverages, where they are and why they mention the brand online. The information helps serve consumers more targeted advertising, which is four times more likely than a regular ad to result in a click.

Coca Cola is increasingly betting on BI, data analytics and AI to drive its strategic business decisions. From its innovative free style fountain machine to finding new ways to engage with customers, Coca Cola is well-equipped to remain at the top of the competition in the future. In a new digital world that is increasingly dynamic, with changing customer behavior, Coca Cola is relying on Big Data to gain and maintain their competitive advantage.

4) American Express GBT

The American Express Global Business Travel company, popularly known as Amex GBT, is an American multinational travel and meetings programs management corporation which operates in over 120 countries and has over 14,000 employees.

Challenges:

Scalability – Creating a single portal for around 945 separate data files from internal and customer systems using the current BI tool would require over 6 months to complete. The earlier tool was used for internal purposes and scaling the solution to such a large population while keeping the costs optimum was a major challenge

Performance – Their existing system had limitations shifting to Cloud. The amount of time and manual effort required was immense

Data Governance – Maintaining user data security and privacy was of utmost importance for Amex GBT

The company was looking to protect and increase its market share by differentiating its core services and was seeking a resource to manage and drive their online travel program capabilities forward. Amex GBT decided to make a strategic investment in creating smart analytics around their booking software.

The solution equipped users to view their travel ROI by categorizing it into three categories cost, time and value. Each category has individual KPIs that are measured to evaluate the performance of a travel plan.

Reducing travel expenses by 30%

Time to Value – Initially it took a week for new users to be on-boarded onto the platform. With Premier Insights that time had now been reduced to a single day and the process had become much simpler and more effective.

Savings on Spends – The product notifies users of any available booking offers that can help them save on their expenditure. It recommends users of possible saving potential such as flight timings, date of the booking, date of travel, etc.

Adoption – Ease of use of the product, quick scale-up, real-time implementation of reports, and interactive dashboards of Premier Insights increased the global online adoption for Amex GBT

5) Airline Solutions Company: BI Accelerates Business Insights

Airline Solutions provides booking tools, revenue management, web, and mobile itinerary tools, as well as other technology, for airlines, hotels and other companies in the travel industry.

Challenge: The travel industry is remarkably dynamic and fast paced. And the airline solution provider’s clients needed advanced tools that could provide real-time data on customer behavior and actions.

They developed an enterprise travel data warehouse (ETDW) to hold its enormous amounts of data. The executive dashboards provide near real-time insights in user-friendly environments with a 360-degree overview of business health, reservations, operational performance and ticketing.

Results: The scalable infrastructure, graphic user interface, data aggregation and ability to work collaboratively have led to more revenue and increased client satisfaction.

6) A specialty US Retail Provider: Leveraging prescriptive analytics

Challenge/Objective: A specialty US Retail provider wanted to modernize its data platform which could help the business make real-time decisions while also leveraging prescriptive analytics. They wanted to discover true value of data being generated from its multiple systems and understand the patterns (both known and unknown) of sales, operations, and omni-channel retail performance.

We helped build a modern data solution that consolidated their data in a data lake and data warehouse, making it easier to extract the value in real-time. We integrated our solution with their OMS, CRM, Google Analytics, Salesforce, and inventory management system. The data was modeled in such a way that it could be fed into Machine Learning algorithms; so that we can leverage this easily in the future.

The customer had visibility into their data from day 1, which is something they had been wanting for some time. In addition to this, they were able to build more reports, dashboards, and charts to understand and interpret the data. In some cases, they were able to get real-time visibility and analysis on instore purchases based on geography!

7) Logistics startup with an objective to become the “Uber of the Trucking Sector” with the help of data analytics

Challenge: A startup specializing in analyzing vehicle and/or driver performance by collecting data from sensors within the vehicle (a.k.a. vehicle telemetry) and Order patterns with an objective to become the “Uber of the Trucking Sector”

Solution: We developed a customized backend of the client’s trucking platform so that they could monetize empty return trips of transporters by creating a marketplace for them. The approach used a combination of AWS Data Lake, AWS microservices, machine learning and analytics.

• Reduced fuel costs
• Optimized Reloads
• More accurate driver / truck schedule planning
• Smarter Routing
• Fewer empty return trips
• Deeper analysis of driver patterns, breaks, routes, etc.

8) Challenge/Objective: A niche segment customer competing against market behemoths looking to become a “Niche Segment Leader”

Solution: We developed a customized analytics platform that can ingest CRM, OMS, Ecommerce, and Inventory data and produce real time and batch driven analytics and AI platform. The approach used a combination of AWS microservices, machine learning and analytics.

• Reduce Customer Churn
• Optimized Order Fulfillment
• More accurate demand schedule planning
• Improve Product Recommendation
• Improved Last Mile Delivery

How can we help you harness the power of data?

At Systems Plus our BI and analytics specialists help you leverage data to understand trends and derive insights by streamlining the searching, merging, and querying of data. From improving your CX and employee performance to predicting new revenue streams, our BI and analytics expertise helps you make data-driven decisions for saving costs and taking your growth to the next level.

Most Popular Blogs

Ready to transform and unlock your full IT potential? Connect with us today to learn more about our comprehensive digital solutions.

Schedule a Consultation

Delivering NOC and SOC IT Managed Services for a Leading Global Entertainment Brand

Elevating user transitions: jml automation mastery at work, saving hundreds of manual hours, smooth transition – navigating a seamless servicenow® upgrade.

TechEnablers Episode 5: Upgrading the In-Store IT Infra

Cyber Program Operations: What might be missing from yo

TechEnablers Episode 4: Transforming IT Service Managem

Driving Efficiency in Retail Logistics

Visualizing Data in Healthcare

Diving into Data and Diversity

AWS Named as a Leader for the 11th Consecutive Year…

Introducing amazon route 53 application recovery controller, amazon sagemaker named as the outright leader in enterprise mlops….

• Made To Order
• Cloud Solutions
• Salesforce Commerce Cloud
• Distributed Agile
• Consulting and Process Optimization
• Data Warehouse & BI
• ServiceNow Consulting and Implementation
• Security Assessment & Mitigation
• Case Studies
• News and Events

Quick Links

• Privacy Statement
•   +91-22-61591100
•   [email protected]

• +49 (0) 40 639 118 91
• info(at)iot-analytics.com
• Contact Form

The top 10 IoT Use Cases

• Adoption of various Internet of Things (IoT) use cases is on the rise, according to the latest 2021 IoT Use Case Adoption report .
• Of the 200+ companies from various industries interviewed, the average company has adopted eight different IoT use cases to date (out of a total of 48).
• IoT use cases related to the smart operations of a company are the most adopted, followed by smart supply chain use cases and use cases involving connected IoT products.
• Read-only remote asset monitoring is the most widely adopted use case (34% of companies interviewed have adopted it).

Why it matters?

• With various IoT topics prioritized by organizations post-COVID, getting a feel for what is hot and what is not is important for both vendors and IoT adopters.
• Vendors should prioritize those use cases that yield a high return on investment (ROI) and those that are on the investment priority list in the coming years.
• IoT adopters can learn from the best practices and examples of existing users to prevent costly mistakes.

The IoT Use Case Adoption Report 2021

In 2021, the average large manufacturing, healthcare, automotive, retail, or energy company has rolled out eight different IoT use cases, according to IoT Analytics’ latest IoT Use Case Adoption Report .

The 430-page report, which is part of IoT Analytics’ ongoing market coverage of IoT applications, is the first such in-depth report and is based on 200+ interviews with IoT end users who have rolled out more than 1,600 IoT projects over the last few years across 48 different IoT use cases. The report shows where companies have been investing and are planning to invest, which industries and regions are ahead, and which use cases promise the highest ROI.

Oil and gas companies and energy companies are ahead of others. They have rolled out an average of 15 use cases. Make no mistake: For many companies, IoT is still a small fraction of their business. The average company we interviewed had $9.6 billion in revenue and only currently spends $33 million on IoT use cases (0.34% of the revenue). The fact that the top use case today has only been adopted by 34% of respondents (but with a predominantly positive ROI) shows how much bigger the IoT opportunity is.

Note: The report looks at IoT use cases that are applicable to most organizations and does not consider industry-specific use cases such as smart heating, ventilation, and air conditioning (HVAC) (applicable only to buildings) or smart transportation systems (applicable to cities). The analysis also does not consider consumer IoT use cases such as Smart Home devices or wearables. For a broader analysis of enterprise IoT applications, see this analysis from 2020. For a deep dive on Smart City IoT use cases, see this an a lysis . A deep dive on Smart Building use cases will be published later this year.

The 10 most adopted IoT Use Cases

Six of the top 10 IoT use cases today (ranked by adoption) aim at making operations smart, thus improving companies’ production processes for manufacturing, enhancing maintenance operations, or advancing any other operations (e.g., energy generation in the case of an energy company, running healthcare operations in the case of a hospital, or running store operations in the case of a retail company). Three of the top 10 use cases are related to smart supply chains, and only one is related to smart products in the field.

Here are the 10 most adopted use cases, ranked by their adoption rate:

1. Read-only Remote Asset Monitoring

Not surprisingly, the simplest IoT use case is also the most adopted. Read-only remote asset monitoring refers to assets that are connected from afar in a read-only manner (i.e., one can visualize the asset data, but one cannot send back any commands to the asset itself). This use case is one of the easiest and cheapest to set up due to its simplicity. In many cases, remote asset monitoring supersedes the error-prone and costly manual task of checking and documenting asset states in person. Adoption in 2020 was clearly accelerated by the pandemic and is expected to grow even further, as 36% of interviewees say they plan to invest significantly in this use case in the next two years.

Adopter’s quote

“In the future, our engineers and plant operations staff will not need to sit in cars to go from their office location to another sub-station or satellite equipment room to perform daily routine activities. Routine read-only activities could be done more frequently, which could indirectly contribute to the reliability of control systems.” Senior operations manager of a mining company in Qatar

Implementation example

Hindustan Coca-Cola Beverages Pvt. Ltd. , located near Pune, India, adopted remote asset monitoring for its PET and can-bottling lines. T https://aiplindia.com/case-study/ he real-time and continuous monitoring of critical parameters (e.g., syrup flow rate, can-rinse pressure, and water temperature) replaced its MS Excel-based manual reporting. In this specific case, the company combined the remote connectivity of its assets with a condition-monitoring use case that generates alarms based on predefined conditions.

| Use case definition (by IoT Analytics)

Remote Asset Monitoring (read-only) = Real-time, one-way data streaming from machinery/equipment to an offsite location (e.g., cloud), making data accessible from anywhere.

2. IoT-based Process Automation

IoT-based process automation has been rolled out by 33% of companies. This type of process automation describes operational processes that were either entirely manual in the past or relied on antiquated, industrial-automation setups but have now been upgraded with state-of-the-art hardware and software. Companies that introduce this use case to upgrade their existing setups often do so to add flexibility and agility in the operations process so that specific process steps can be changed in the future. This becomes important because companies are increasingly interested in aligning their manufacturing and operations processes to ever-changing customer demands.

“Our manufacturing processes used to have many paper-based steps involved. IoT process automation has reduced many of these paper-based processes, resulting in a reduction in the number of errors, labour cost and real-time access to data.” CIO of a food and beverage manufacturer in Canada

An Australian farmer , reported he was able to use 20% less water thanks to a new IoT-based irrigation system supplied by Lindsay , a US-based manufacturer of pivot-irrigation systems. Farmers increasingly adopt smart IoT-based irrigation systems to automate the process of irrigation and thereby enhance their crop yields.

IoT-based Process Automation = The use of real-time data from connected assets to improve an entire operational process (e.g., a manufacturing process).

3. Remote Asset Monitoring and Control (read/write)

“Read/Write remote asset monitoring and control” is the extension of “Read-only remote asset monitoring.” On top of “reading” the assets data, with this setup one can also communicate back and thereby influence asset control from afar (i.e., “writing” data back).

Just like the read-only asset monitoring use case, this use case was accelerated by the COVID -19 pandemic as service teams, engineers, and other staff needed to find ways to reach an asset that needed their attention while being in a different location.

Because of the added complexity and security risk of controlling an asset, on top of just simply monitoring, this use case comes with significantly higher costs of installation and maintenance. However, it has been proven that such solutions pay off in a relatively short time: Of those decision makers whom IoT Analytics interviewed, 51% reported amortization in less than 24 months.

“Remote monitoring and control of freezers has saved costly materials before being ruined.” Research and development (R&D) manager of a manufacturing company in UK

Schlumberger , an oilfield service company, recently adopted a monitoring and control solution from Advantech and ReStream . These companies had partnered up to supply and develop a new oilfield fluid-monitoring system based on LTE connectivity. The project helped to achieve flow assurance, ensure asset integrity, and optimize production. It also helped to increase labor safety. Oilfield companies suffer from a dynamic chemical environment that, if unchecked, can lead to premature wear, lost productivity, and the release of lethal poisonous gases.

Remote Asset Monitoring and Control (read/write) = Real-time, two-way data streaming of machinery/equipment from/to an offsite location, enabling data access and the sending of commands from anywhere.

4. Vehicle Fleet Management (track/trace)

The management of vehicles in a fleet is the number one IoT supply chain use case right now. The larger the fleet of trucks (or other means of transport) to manage, the higher the complexity. Cross-border fleet management can be especially overwhelming, which is why 31% of companies have rolled out a professional vehicle fleet management solution to gather real-time information.

Today, most fleet management solutions are reliant on wide-area connectivity, such as cellular (2G, 3G, 4G). Several new, low-cost IoT satellite technologies have recently been launched and promise to add another option to this use case so that vehicle fleets are always truly connected at a reasonable price. New companies include Hiber (IoT-focused satellite network) and Starlink (part of SpaceX). Both plan to send thousands of new satellites into space in the coming years to provide low-latency, high-bandwidth connectivity globally.

“Now we know virtually in real time where our wagons are located, which routes we can optimize, and where there is potential for savings for our customers.” Suzy Verachten, project leader, Lineas (European Rail Freight Operator)

Lineas , the largest private rail freight operator in Europe, managed to increase capacity utilization of its fleet by more than 40 percent by implementing the Bosch fleet management solution.

Vehicle Fleet Management (track/trace) = Tracking and tracing the location of individual vehicles either on site while in transit or as general fleet management.

5. Location Tracking (e.g., GPS)

Not every company sells smart-connected IoT products, but of those that do, tracking the location of the asset is the number one use case (31% of companies in our interview set have adopted it). The results from this interview series are in line with results from the IoT Commercialization & Business Model Report 2020 , which found that geolocalization was also the number one use case in terms of customer adoption when original equipment manufacturers (OEMs) were asked which smart product features their end customers were adopting as part of their IoT-enabled assets. Location tracking is therefore important for developing a successful IoT business model .

Tracking the location of an asset can be beneficial to both the vendor of the products (e.g., by understanding usage patterns) and to the user (e.g., by finding a lost item or by mitigating theft).

“Connectivity means we can monitor distance covered, send service alerts, or lock the bike remotely. Things become even more interesting when we mesh this data with other transport data. If we can map the routes of thousands of cyclists, could we help urban planners develop better cycling infrastructure?” Taco Carlier, co-founder, VanMoof

VanMoof, an Amsterdam-based e-bike manufacturer, offers an anti-theft service to its customers that is based on the location tracking of a bike. For a monthly fee, the company sends its “bike hunters” if a customer’s bike is stolen. The bikes are connected and tracked by a combination of Bluetooth and cellular networks. This allows the company to keep its promise of recovering the bike in up to 70% of cases.

Location Tracking (e.g., GPS) = Tracking product location to determine its movement and geographic position.

6. IoT for Asset/Plant Performance Optimization

Asset performance management (APM) is a term in manufacturing to describe methods of capturing and integrating data, visualizing it, and analyzing it to improve the reliability and availability of physical plant–floor assets. IoT for asset/plant performance optimization is the modern version of APM. It integrates state-of-the-art data-capture and integration tools (like IoT gateways) and software tools (like IoT platforms) to analyze how assets can be run and maintained at optimum levels (e.g., optimized asset speed settings, optimized material input settings, or optimized maintenance intervals).

More than 30% of companies in our interview set have adopted IoT for asset/plant performance optimization. With 42% of companies planning to invest significantly in IoT for this purpose, it is among the use cases with the highest expected growth rate.

“The use of IoT for plant performance optimization has been a success since it now allows us to run our CHP power utilities at optimal levels based on variable fuel mix and plant loads for steam and power.” CEO, manufacturing: chemicals and chemical products , India

A beverage manufacturer that was running mostly older equipment used an IoT solution provided by Relayr in one of its bottling plants, which produces 1.2 million bottles per day per production line. It achieved improvements of 11% in performance and 8% in quality. Implementing an IoT retrofit kit and an IoT gateway, the company was able to identify the processes responsible for 90% of the plant’s downtime. Based on the IoT data collected from the legacy machinery of different manufacturers, the company was able to address several issues (e.g., changing the control logic of the machines).

IoT for Asset/Plant Performance Optimization = The use of real-time data from connected assets to optimize individual asset health for better operations performance at a site.

7. IoT- based Quality Control & Management

IoT-based quality control and management involves the use of machine vision or other IoT sensor data to detect quality issues in real time during operations. The use case pays off particularly quickly: Of the 30% of companies that have implemented this IoT use case, two-thirds report amortization in fewer than 24 months.

“We can now tell quickly if the environmental parameters are not quite right at some point, either within the paint shop or in one of the buffer areas. It takes a lot of data to do this, which we collect throughout the process, evaluate historically and analyse in real time” Martin Hilt, innovation and digitalization officer at BMW

A leading global automobile manufacturer automated the testing of the electronic functionality of its cars during the manufacturing process. As the company was adding greater electronic functionality to the cars to stay current with technological developments, it was looking to find ways to optimize its time-consuming, manual method of checking the operation of electronic parts. With the help of Cognizant , the company was able to completely automate vehicle testing and also implement a mixed-mode validation to simulate missing parts that would arrive late due to unforeseen supply chain issues. The solution realized a positive return on investment in four months.

IoT- based Quality Control & Management = The use of IoT sensor data or machine vision to monitor operational process parameters and detect faults (in real time) with the goal of reducing scrap/rework.

8. IoT-based goods condition monitoring in transit

Monitoring the conditions of goods is essential in industries such as pharmaceuticals or food and beverage and can even be seen as a way to help tackle the global food shortage. Temperature sensor data is one of the most important sensor values for this use case; it guarantees the safety of the final product by close monitoring throughout the entire supply chain, both in transit and in storage.

Condition-monitoring solutions were used by 29% of the interviewed decision makers as part of their digital supply chain initiatives.

“The assessment [of goods in transit] was an eye-opener, because we never looked at supply chain risk holistically before. Now we know where the risks are.” An D’haenens, logistics manager EMEA, DuPont

A global life sciences company worked with DHL to enable pallets with smart sensors. The sensors are equipped with low-power wide-area (LPWA) connectivity and track geographic location, movement, delay, shock, and temperature. This ability has enabled the life science company to reduce its life-cycle costs by 6 to 7%, with reported reliability of nearly 100%.

IoT-based goods condition monitoring in transit = The use of IoT sensor data to monitor the condition of transported goods and ensure they remain safe and are not exposed to the wrong temperatures or shocks.

9. Predictive Maintenance

To forecast the remaining useful life of assets and ensure they are repaired before they fail, 29% of all companies have invested in solutions at the intersection of artificial intelligence and maintenance. As IoT Analytics highlighted in a recent blog post, predictive maintenance (PdM) is an en vogue topic, with 40% of interviewed decision makers planning to invest significant amounts of money in the coming two years.

“PdM enabled us to actively monitor highly critical assets, provided us with multiple feeds of various data points (temp, vibration, throughput), allowed [us] to perform analysis of the asset health in real time, and helped us to prevent the need [for] reactive maintenance actions for critical assets.” IT director at an Oil&Gas company in the United States

Colgate-Palmolive Company , a US-based consumer goods manufacturer, leveraged a predictive maintenance solution to optimize its manufacturing capacity and prevent costly downtime. It installed wireless sensors on 2,000 machines (including tube makers and liquid machines) to collect machine data. The predictive maintenance solution detected increased temperatures and alerted staff, thereby preventing nearly 200 hours of downtime and a missing output of 2.8 million tubes of toothpaste.

Predictive Maintenance = The use of real-time IoT sensor data and artificial intelligence (AI) techniques to determine when maintenance should be performed on specific equipment.

10. On-Site Track and Trace

The availability of low-cost sensors and trackers has made it easy and relatively cheap to track goods and tools at construction sites, ports, and even inside buildings (e.g., factories). Of the experts interviewed, 29% have implemented such a solution. The median spending for a fully functional, on-site track-and-trace solution (in our interview set) is only $25,000—one of the lowest in the data set. Costs can scale up, however, as more and more assets get connected.

“Having On-Site Track & Trace has improved our efficiency [and] reduced labor time for tracking items (we have almost 1,000 stock-keeping units). We are now also able to check stock balances in real time.” Head of IT at a retailer in Singapore

Continental , a German automotive supplier, implemented KINEXON’S real-time locating solution at its logistics center and production site in Regensburg. More than 50 transportation vehicles and more than 3,000 load carriers are tracked live inside the 30,000 m² building sites, allowing Continental to locate each of these items within a precision of several centimeters (inches), thereby creating a transparent material flow in the supply chain and increasing overall throughput.

On-Site Track and Trace = Tracking and tracing goods while on site (e.g., at production sites, ports, logistics centers, or warehouses).

More information and further reading

Are you interested in learning more about iot use cases.

The IoT Use Case Adoption Report 2021 is a comprehensive 430-page report examining the adoption of 48 IoT use cases across 4 types: smart operations, smart supply chain, connected products, and connected transport. It includes the adoption (by region/country, industry), a detailed breakdown of spending, ROI per use case, key vendors & vendor satisfaction, as well as best practices and lessons learnt. It is part of IoT Analytics’ ongoing coverage of IoT in general .

By loading the video, you agree to YouTube’s privacy policy. Learn more

Always unblock YouTube

This report provides answers to the following questions (among others):

• What are the most adopted IoT use cases (out of a list of 48 use cases)?
• Which IoT use cases have the highest ROI?
• How much do companies spend to implement IoT use cases?
• How much do they spend on hardware, software and services respectively?
• Which IoT use cases do companies plan to invest in in the next 2 years?
• Who are the leading vendors for IoT use cases and how satisfied are end-users?
• What supporting technologies do IoT end-users implement as part of their IoT use case projects?
• What are the best practices and lessons learned of companies that have adopted the IoT?
• And more…

The sample of the report gives you a holistic overview of the available analysis (outline, key slides). The sample also provides additional context on the topic and describes the methodology of the analysis. You can download the sample here :

Related articles

You may also be interested in the following recent articles:

• State of IoT 2021: Number of connected IoT devices growing 9% to 12.3 billion globally, cellular IoT now surpassing 2 billion
• IoT technology market attractiveness: Where to invest going into 2022
• Top 10 IoT applications in 2020
• The top 10 Smart City use cases that are being prioritized now
• How to create a successful IoT business model – insights from early innovators
• Predictive Maintenance Market: The Evolution from Niche Topic to High ROI Application

Are you interested in continued IoT coverage and updates?

Subscribe to our  newsletter  and follow us on  LinkedIn  and  Twitter  to stay up-to-date on the latest trends shaping the IoT markets. For complete enterprise IoT coverage with access to all of IoT Analytics’ paid content & reports including dedicated analyst time check out   Enterprise subscription . 

Share this with others:

Philipp Wegner

Iot research newsletter.

Sign up for our exclusive email updates today, and receive the latest market insights before others.

Most Recent Articles

• Benchmarking IoT mobile operator pricing: MNOs vs. MVNOs
• IoT Startup Landscape 2024: 7 notable insights
• The top 6 paradigms shaping the factory of the future—insights from 500 manufacturers
• Global cellular IoT connectivity market reached $15B in 2023, 5G set to drive further growth
• IoT market update: Enterprise IoT market size reached $269 billion in 2023, with growth deceleration in 2024

Search the Blog

Blog categories.

• Industrial IoT
• Non-industrial IoT applications
• IoT Platforms & Software
• IoT Connectivity & Hardware
• General IoT

IoT Analytics, founded and operating out of Germany, is a leading provider of strategic IoT market insights and a trusted advisor for 1000+ corporate partners worldwide.

Learn more about how we can help you achieve your goals faster with the right data-driven insights and intelligence.

We need your consent before you can continue on our website. We use cookies and other technologies on our website. Some of them are essential, while others help us to improve this website and your experience. You can find more information about the use of your data in our privacy policy .

• External Media

Accept only essential cookies

Individual Privacy Preferences

Cookie Details Privacy Policy Legal Notice

We use cookies and other technologies on our website. Some of them are essential, while others help us to improve this website and your experience. You can find more information about the use of your data in our privacy policy . Here you will find an overview of all cookies used. You can give your consent to whole categories or display further information and select certain cookies.

Accept all Save Accept only essential cookies

Essential cookies enable basic functions and are necessary for the proper function of the website.

Show Cookie Information Hide Cookie Information

Statistics cookies collect information anonymously. This information helps us to understand how our visitors use our website.

Marketing cookies are used by third-party advertisers or publishers to display personalized ads. They do this by tracking visitors across websites.

Content from video platforms and social media platforms is blocked by default. If External Media cookies are accepted, access to those contents no longer requires manual consent.

Privacy Policy Legal Notice

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Author Biographies
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Acceptance of an iot system for strawberry cultivation: a case study of different users.

1. Introduction

2. related works, 3. materials and methods, 3.1. diagnosis, 3.1.1. initial conditions, 3.1.2. participants, 3.2. components, 3.3. connections, 3.4. programming, 3.5. web application, 3.6. instruments, 4.1. validation, 4.2. acceptance, 4.3. interview, 4.3.1. technical user, 4.3.2. administrator, 5. discussion, 6. conclusions, 6.1. limitations, 6.2. future works, author contributions, institutional review board statement, informed consent statement, data availability statement, acknowledgments, conflicts of interest.

• Moore, S.J.; Nugent, C.D.; Zhang, S.; Cleland, I. IoT Reliability: A Review Leading to 5 Key Research Directions. CCF Trans. Pervasive Comput. Interact. 2020 , 2 , 147–163. [ Google Scholar ] [ CrossRef ]
• Abolhassani Khajeh, S.; Saberikamarposhti, M.; Rahmani, A.M. Real-Time Scheduling in IoT Applications: A Systematic Review. Sensors 2022 , 23 , 232. [ Google Scholar ] [ CrossRef ]
• Chataut, R.; Phoummalayvane, A.; Akl, R. Unleashing the Power of IoT: A Comprehensive Review of IoT Applications and Future Prospects in Healthcare, Agriculture, Smart Homes, Smart Cities, and Industry 4.0. Sensors 2023 , 23 , 7194. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Adli, H.K.; Remli, M.A.; Wan Salihin Wong, K.N.S.; Ismail, N.A.; González-Briones, A.; Corchado, J.M.; Mohamad, M.S. Recent Advancements and Challenges of AIoT Application in Smart Agriculture: A Review. Sensors 2023 , 23 , 3752. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Mirani, A.A.; Velasco-Hernandez, G.; Awasthi, A.; Walsh, J. Key Challenges and Emerging Technologies in Industrial IoT Architectures: A Review. Sensors 2022 , 22 , 5836. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Mu, X.; Antwi-Afari, M.F. The Applications of Internet of Things (IoT) in Industrial Management: A Science Mapping Review. Int. J. Prod. Res. 2024 , 62 , 1928–1952. [ Google Scholar ] [ CrossRef ]
• Whaiduzzaman, M.; Barros, A.; Chanda, M.; Barman, S.; Sultana, T.; Rahman, M.S.; Roy, S.; Fidge, C. A Review of Emerging Technologies for IoT-Based Smart Cities. Sensors 2022 , 22 , 9271. [ Google Scholar ] [ CrossRef ]
• Alsharif, M.H.; Jahid, A.; Kelechi, A.H.; Kannadasan, R. Green IoT: A Review and Future Research Directions. Symmetry 2023 , 15 , 757. [ Google Scholar ] [ CrossRef ]
• Aldin, H.N.S.; Ghods, M.R.; Nayebipour, F.; Torshiz, M.N. A Comprehensive Review of Energy Harvesting and Routing Strategies for IoT Sensors Sustainability and Communication Technology. Sens. Int. 2024 , 5 , 100258. [ Google Scholar ] [ CrossRef ]
• Zulkifli, C.Z.; Garfan, S.; Talal, M.; Alamoodi, A.H.; Alamleh, A.; Ahmaro, I.Y.Y.; Sulaiman, S.; Ibrahim, A.B.; Zaidan, B.B.; Ismail, A.R.; et al. IoT-Based Water Monitoring Systems: A Systematic Review. Water 2022 , 14 , 3621. [ Google Scholar ] [ CrossRef ]
• Falcone, R.; Sapienza, A. On the Users’ Acceptance of IoT Systems: A Theoretical Approach. Information 2018 , 9 , 53. [ Google Scholar ] [ CrossRef ]
• Alkawsi, G.A.; Ali, N.B. A Systematic Review of Individuals’ Acceptance of IOT-Based Technologies. Int. J. Eng. Technol. 2018 , 7 , 136. [ Google Scholar ] [ CrossRef ]
• Tsourela, M.; Nerantzaki, D.-M. An Internet of Things (IoT) Acceptance Model. Assessing Consumer’s Behavior toward IoT Products and Applications. Futur. Internet 2020 , 12 , 191. [ Google Scholar ] [ CrossRef ]
• Suhluli, S.A.; Ali Khan, S.M.F. Determinants of User Acceptance of Wearable IoT Devices. Cogent Eng. 2022 , 9 , 2087456. [ Google Scholar ] [ CrossRef ]
• Chaurasia, N.; Kumar, P. A Comprehensive Study on Issues and Challenges Related to Privacy and Security in IoT. e-Prime Adv. Electr. Eng. Electron. Energy 2023 , 4 , 100158. [ Google Scholar ] [ CrossRef ]
• Robles-Gomez, A.; Tobarra, L.; Pastor-Vargas, R.; Hernandez, R.; Haut, J.M. Analyzing the Users’ Acceptance of an IoT Cloud Platform Using the UTAUT/TAM Model. IEEE Access 2021 , 9 , 150004–150020. [ Google Scholar ] [ CrossRef ]
• Almaiah, M.; Alfaisal, R.; Salloum, S.; Al-Otaibi, S.; Shishakly, R.; Lutfi, A.; Alrawad, M.; Mulhem, A.; Awad, A.; Al-Maroof, R. Integrating Teachers’ TPACK Levels and Students’ Learning Motivation, Technology Innovativeness, and Optimism in an IoT Acceptance Model. Electronics 2022 , 11 , 3197. [ Google Scholar ] [ CrossRef ]
• Liu, D.; Li, Q.; Han, S. Using Extended Technology Acceptance Model to Assess the Adopt Intention of a Proposed IoT-Based Health Management Tool. Sensors 2022 , 22 , 6092. [ Google Scholar ] [ CrossRef ]
• Ben Arfi, W.; Ben Nasr, I.; Khvatova, T.; Ben Zaied, Y. Understanding Acceptance of EHealthcare by IoT Natives and IoT Immigrants: An Integrated Model of UTAUT, Perceived Risk, and Financial Cost. Technol. Forecast. Soc. Change 2021 , 163 , 120437. [ Google Scholar ] [ CrossRef ]
• Chaiyason, S.; Wisaeng, K. Efficiency, Acceptance and SWOT Analysis of IoT Electrical Switchboards: A TAM Perspective. Int. J. Comput. Digit. Syst. 2022 , 12 , 557–568. [ Google Scholar ] [ CrossRef ]
• Walczak, R.; Koszewski, K.; Olszewski, R.; Ejsmont, K.; Kálmán, A. Acceptance of IoT Edge-Computing-Based Sensors in Smart Cities for Universal Design Purposes. Energies 2023 , 16 , 1024. [ Google Scholar ] [ CrossRef ]
• Xu, J.; Gu, B.; Tian, G. Review of Agricultural IoT Technology. Artif. Intell. Agric. 2022 , 6 , 10–22. [ Google Scholar ] [ CrossRef ]
• Farooq, H.; Rehman, H.U.; Javed, A.; Shoukat, M.; Dudely, S. A Review on Smart IoT Based Farming. Ann. Emerg. Technol. Comput. 2020 , 4 , 17–28. [ Google Scholar ] [ CrossRef ]
• Cáceres, L.; Teneda, E. Acidity Analysis in Different Blackberry Dilutions Using IoT. Int. J. Eng. Insights 2023 , 1 , 13–18. [ Google Scholar ] [ CrossRef ]
• Qureshi, T.; Saeed, M.; Ahsan, K.; Malik, A.A.; Muhammad, E.S.; Touheed, N. Smart Agriculture for Sustainable Food Security Using Internet of Things (IoT). Wirel. Commun. Mob. Comput. 2022 , 2022 , 9608394. [ Google Scholar ] [ CrossRef ]
• Farooq, M.S.; Riaz, S.; Abid, A.; Umer, T.; Zikria, Y. Bin Role of IoT Technology in Agriculture: A Systematic Literature Review. Electronics 2020 , 9 , 319. [ Google Scholar ] [ CrossRef ]
• Dhanaraju, M.; Chenniappan, P.; Ramalingam, K.; Pazhanivelan, S.; Kaliaperumal, R. Smart Farming: Internet of Things (IoT)-Based Sustainable Agriculture. Agriculture 2022 , 12 , 1745. [ Google Scholar ] [ CrossRef ]
• Farooq, M.S.; Javid, R.; Riaz, S.; Atal, Z. IoT Based Smart Greenhouse Framework and Control Strategies for Sustainable Agriculture. IEEE Access 2022 , 10 , 99394–99420. [ Google Scholar ] [ CrossRef ]
• Calicioglu, O.; Flammini, A.; Bracco, S.; Bellù, L.; Sims, R. The Future Challenges of Food and Agriculture: An Integrated Analysis of Trends and Solutions. Sustainability 2019 , 11 , 222. [ Google Scholar ] [ CrossRef ]
• Akhter, R.; Sofi, S.A. Precision Agriculture Using IoT Data Analytics and Machine Learning. J. King Saud Univ.—Comput. Inf. Sci. 2022 , 34 , 5602–5618. [ Google Scholar ] [ CrossRef ]
• Singh, R.K.; Berkvens, R.; Weyn, M. AgriFusion: An Architecture for IoT and Emerging Technologies Based on a Precision Agriculture Survey. IEEE Access 2021 , 9 , 136253–136283. [ Google Scholar ] [ CrossRef ]
• Navarro, E.; Costa, N.; Pereira, A. A Systematic Review of IoT Solutions for Smart Farming. Sensors 2020 , 20 , 4231. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Sharef, B.T. The Usage of Internet of Things in Agriculture: The Role of Size and Perceived Value. Informatica 2022 , 46 , 73. [ Google Scholar ] [ CrossRef ]
• Mizukami, Y.; Nakano, J. International Comparison of Cross-Disciplinary Integration in Industry 4.0: A Co-Authorship Analysis Using Academic Literature Databases. PLoS ONE 2022 , 17 , e0275306. [ Google Scholar ] [ CrossRef ]
• Puntel, L.A.; Bolfe, É.L.; Melchiori, R.J.M.; Ortega, R.; Tiscornia, G.; Roel, A.; Scaramuzza, F.; Best, S.; Berger, A.G.; Hansel, D.S.S.; et al. How Digital Is Agriculture in a Subset of Countries from South America? Adoption and Limitations. Crop Pasture Sci. 2022 , 74 , 555–572. [ Google Scholar ] [ CrossRef ]
• Gumz, J.; Fettermann, D.C.; Sant’Anna, Â.M.O.; Tortorella, G.L. Social Influence as a Major Factor in Smart Meters’ Acceptance: Findings from Brazil. Results Eng. 2022 , 15 , 100510. [ Google Scholar ] [ CrossRef ]
• Alvear-Puertas, V.E.; Burbano-Prado, Y.A.; Rosero-Montalvo, P.D.; Tözün, P.; Marcillo, F.; Hernandez, W. Smart and Portable Air-Quality Monitoring IoT Low-Cost Devices in Ibarra City, Ecuador. Sensors 2022 , 22 , 7015. [ Google Scholar ] [ CrossRef ]
• Espinoza, M.G.T.; Melendrez, J.R.N.; Clemente, L.A.N. A Survey and an IoT Cybersecurity Recommendation for Public and Private Hospitals in Ecuador. Adv. Sci. Technol. Eng. Syst. J. 2020 , 5 , 518–528. [ Google Scholar ] [ CrossRef ]
• Nawi, N.C.; Al Mamun, A.; Md Nasir, N.A.; Rahman, M.K. Analyzing Customer Acceptance of the Internet of Things (IoT) in the Retail Industry. J. Ambient Intell. Humaniz. Comput. 2023 , 14 , 5225–5237. [ Google Scholar ] [ CrossRef ]
• Romero-Riaño, E.; Galeano-Barrera, C.; Guerrero, C.D.; Martinez-Toro, M.; Rico-Bautista, D. IoT Applied to Irrigation Systems in Agriculture: A Usability Analysis. Rev. Colomb. Comput. 2022 , 23 , 44–52. [ Google Scholar ] [ CrossRef ]
• Tohidyan Far, S.; Rezaei-Moghaddam, K. Determinants of Iranian Agricultural Consultants’ Intentions toward Precision Agriculture: Integrating Innovativeness to the Technology Acceptance Model. J. Saudi Soc. Agric. Sci. 2017 , 16 , 280–286. [ Google Scholar ] [ CrossRef ]
• Hanson, E.D.; Cossette, M.K.; Roberts, D.C. The Adoption and Usage of Precision Agriculture Technologies in North Dakota. Technol. Soc. 2022 , 71 , 102087. [ Google Scholar ] [ CrossRef ]
• Lin, H.-C.; Chang, T.-Y.; Kuo, S.-H. Effects of Social Influence and System Characteristics on Traceable Agriculture Product Reuse Intention of Elderly People: Integrating Trust and Attitude Using the Technology Acceptance Model. J. Res. Educ. Sci. 2018 , 63 , 291–319. [ Google Scholar ] [ CrossRef ]
• Khoza, S.; de Beer, L.T.; van Niekerk, D.; Nemakonde, L. A Gender-Differentiated Analysis of Climate-Smart Agriculture Adoption by Smallholder Farmers: Application of the Extended Technology Acceptance Model. Gend. Technol. Dev. 2021 , 25 , 1–21. [ Google Scholar ] [ CrossRef ]
• McDonald, N.; Fogarty, E.S.; Cosby, A.; McIlveen, P. Technology Acceptance, Adoption and Workforce on Australian Cotton Farms. Agriculture 2022 , 12 , 1180. [ Google Scholar ] [ CrossRef ]
• Anas, S.A.B.; Singh, R.S.S.; Kamarudin, N.A.B. Designing an IoT Agriculture Monitoring System for Improving Farmer’s Acceptance of Using IoT Technology. Eng. Technol. Appl. Sci. Res. 2022 , 12 , 8157–8163. [ Google Scholar ] [ CrossRef ]
• Thomas, R.J.; O’Hare, G.; Coyle, D. Understanding Technology Acceptance in Smart Agriculture: A Systematic Review of Empirical Research in Crop Production. Technol. Forecast. Soc. Change 2023 , 189 , 122374. [ Google Scholar ] [ CrossRef ]
• M5STACK CORE2. Available online: http://docs.m5stack.com/en/core/core2 (accessed on 2 January 2024).
• Aosong Electronics Co., Ltd. Digital-Output Relative Humidity & Temperature Sensor/Module. Available online: https://www.sparkfun.com/datasheets/Sensors/Temperature/DHT22.pdf (accessed on 30 May 2024).
• DatasheetHub. FC-28 Soil Moisture Sensor Module. Available online: https://www.datasheethub.com/fc-28-soil-moisture-sensor-module/ (accessed on 30 May 2024).
• LS ELECTRIC MC-12B 230VAC 1A1B. Available online: https://www.tme.eu/es/details/mc-12b-230vac/contactores-modulos-principales/ls-electric/mc-12b-230vac-1a1b/ (accessed on 30 May 2024).
• Gao, L.; Bai, X. A Unified Perspective on the Factors Influencing Consumer Acceptance of Internet of Things Technology. Asia Pacific J. Mark. Logist. 2014 , 26 , 211–231. [ Google Scholar ] [ CrossRef ]
• Bilal, M.; Tayyab, M.; Hamza, A.; Shahzadi, K.; Rubab, F. The Internet of Things for Smart Farming: Measuring Productivity and Effectiveness. Eng. Proc. 2023 , 58 , 106. [ Google Scholar ]
• Rajak, P.; Ganguly, A.; Adhikary, S.; Bhattacharya, S. Internet of Things and Smart Sensors in Agriculture: Scopes and Challenges. J. Agric. Food Res. 2023 , 14 , 100776. [ Google Scholar ] [ CrossRef ]

Click here to enlarge figure

Share and Cite

Varela-Aldás, J.; Gavilanes, A.; Velasco, N.; Del-Valle-Soto, C.; Bran, C. Acceptance of an IoT System for Strawberry Cultivation: A Case Study of Different Users. Sustainability 2024 , 16 , 7221. https://doi.org/10.3390/su16167221

Varela-Aldás J, Gavilanes A, Velasco N, Del-Valle-Soto C, Bran C. Acceptance of an IoT System for Strawberry Cultivation: A Case Study of Different Users. Sustainability . 2024; 16(16):7221. https://doi.org/10.3390/su16167221

Varela-Aldás, José, Alex Gavilanes, Nancy Velasco, Carolina Del-Valle-Soto, and Carlos Bran. 2024. "Acceptance of an IoT System for Strawberry Cultivation: A Case Study of Different Users" Sustainability 16, no. 16: 7221. https://doi.org/10.3390/su16167221

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

Causal Deep Learning for the Detection of Adverse Drug Reactions: Drug-Induced Acute Kidney Injury as a Case Study

Affiliations.

• 1 Sorbonne Université, INSERM, Univ Paris 13, Laboratoire d'Informatique Médicale et d'Ingénierie des Connaissances pour la eSanté, LIMICS, F-75006 Paris, France.
• 2 Institute of Applied Biosciences, Centre for Research and Development Hellas, Thessaloniki, Greece.
• PMID: 39176914
• DOI: 10.3233/SHTI240533

Causal Deep/Machine Learning (CDL/CML) is an emerging Artificial Intelligence (AI) paradigm. The combination of causal inference and AI could mine explainable causal relationships between data features, providing useful insights for various applications, e.g. Pharmacovigilance (PV) signal detection upon Real-World Data. The objective of this study is to demonstrate the use of CDL for potential PV signal validation using Electronic Health Records as input data source.

Keywords: causal machine learning; drug safety; electronic health records.

PubMed Disclaimer

• Search in MeSH
• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 21 August 2024

Loss of plasticity in deep continual learning

• Shibhansh Dohare   ORCID: orcid.org/0000-0002-3796-9347 1 ,
• J. Fernando Hernandez-Garcia 1 ,
• Qingfeng Lan   ORCID: orcid.org/0000-0001-8568-7603 1 ,
• Parash Rahman   ORCID: orcid.org/0000-0001-8430-2679 1 ,
• A. Rupam Mahmood   ORCID: orcid.org/0000-0001-6266-162X 1 , 2 &
• Richard S. Sutton 1 , 2  

Nature volume  632 ,  pages 768–774 ( 2024 ) Cite this article

3074 Accesses

91 Altmetric

Metrics details

• Computer science
• Human behaviour

Artificial neural networks, deep-learning methods and the backpropagation algorithm 1 form the foundation of modern machine learning and artificial intelligence. These methods are almost always used in two phases, one in which the weights of the network are updated and one in which the weights are held constant while the network is used or evaluated. This contrasts with natural learning and many applications, which require continual learning. It has been unclear whether or not deep learning methods work in continual learning settings. Here we show that they do not—that standard deep-learning methods gradually lose plasticity in continual-learning settings until they learn no better than a shallow network. We show such loss of plasticity using the classic ImageNet dataset and reinforcement-learning problems across a wide range of variations in the network and the learning algorithm. Plasticity is maintained indefinitely only by algorithms that continually inject diversity into the network, such as our continual backpropagation algorithm, a variation of backpropagation in which a small fraction of less-used units are continually and randomly reinitialized. Our results indicate that methods based on gradient descent are not enough—that sustained deep learning requires a random, non-gradient component to maintain variability and plasticity.

Similar content being viewed by others

Inferring neural activity before plasticity as a foundation for learning beyond backpropagation

Three types of incremental learning

Meta-learning biologically plausible plasticity rules with random feedback pathways

Machine learning and artificial intelligence have made remarkable progress in the past decade, with landmark successes in natural-language processing 2 , 3 , biology 4 , game playing 5 , 6 , 7 , 8 and robotics 9 , 10 . All these systems use artificial neural networks, whose computations are inspired by the operation of human and animal brains. Learning in these networks refers to computational algorithms for changing the strengths of their connection weights (computational synapses). The most important modern learning methods are based on stochastic gradient descent (SGD) and the backpropagation algorithm, ideas that originated at least four decades ago but are much more powerful today because of the availability of vastly greater computer power. The successes are also because of refinements of the learning and training techniques that together make the early ideas effective in much larger and more deeply layered networks. These methodologies are collectively referred to as deep learning.

Despite its successes, deep learning has difficulty adapting to changing data. Because of this, in almost all applications, deep learning is restricted to a special training phase and then turned off when the network is actually used. For example, large language models such as ChatGPT are trained on a large generic training set and then fine-tuned on smaller datasets specific to an application or to meet policy and safety goals, but finally their weights are frozen before the network is released for use. With current methods, it is usually not effective to simply continue training on new data when they become available. The effect of the new data is either too large or too small and not properly balanced with the old data. The reasons for this are not well understood and there is not yet a clear solution. In practice, the most common strategy for incorporating substantial new data has been simply to discard the old network and train a new one from scratch on the old and new data together 11 , 12 . When the network is a large language model and the data are a substantial portion of the internet, then each retraining may cost millions of dollars in computation. Moreover, a wide range of real-world applications require adapting to change. Change is ubiquitous in learning to anticipate markets and human preferences and in gaming, logistics and control systems. Deep-learning systems would be much more powerful if they, like natural-learning systems, were capable of continual learning.

Here we show systematically that standard deep-learning methods lose their ability to learn with extended training on new data, a phenomenon that we call loss of plasticity. We use classic datasets, such as ImageNet and CIFAR-100, modified for continual learning, and standard feed-forward and residual networks with a wide variety of standard learning algorithms. Loss of plasticity in artificial neural networks was first shown at the turn of the century in the psychology literature 13 , 14 , 15 , before the development of deep-learning methods. Plasticity loss with modern methods was visible in some recent works 11 , 16 , 17 , 18 and most recently has begun to be explored explicitly 12 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 . Loss of plasticity is different from catastrophic forgetting, which concerns poor performance on old examples even if they are not presented again 28 , 29 , 30 .

Although standard deep-learning methods lose plasticity with extended learning, we show that a simple change enables them to maintain plasticity indefinitely in both supervised and reinforcement learning. Our new algorithm, continual backpropagation, is exactly like classical backpropagation except that a tiny proportion of less-used units are reinitialized on each step much as they were all initialized at the start of training. Continual backpropagation is inspired by a long history of methods for automatically generating and testing features, starting with Selfridge’s Pandemonium in 1959 (refs.  19 , 20 , 31 , 32 , 33 , 34 , 35 ). The effectiveness of continual backpropagation shows that the problem of plasticity loss is not inherent in artificial neural networks.

Plasticity loss in supervised learning

The primary purpose of this article is to demonstrate loss of plasticity in standard deep-learning systems. For the demonstration to be convincing, it must be systematic and extensive. It must consider a wide range of standard deep-learning networks, learning algorithms and parameter settings. For each of these, the experiments must be run long enough to expose long-term plasticity loss and be repeated enough times to obtain statistically significant results. Altogether, more computation is needed by three or four orders of magnitude compared with what would be needed to train a single network. For example, a systematic study with large language models would not be possible today because just a single training run with one of these networks would require computation costing millions of dollars. Fortunately, advances in computer hardware have continued apace since the development of deep learning and systematic studies have become possible with the deep-learning networks used earlier and with some of the longer-lived test problems. Here we use ImageNet, a classic object-recognition test bed 36 , which played a pivotal role in the rise of deep learning 37 and is still influential today.

The ImageNet database comprises millions of images labelled by nouns (classes) such as types of animal and everyday object. The typical ImageNet task is to guess the label given an image. The standard way to use this dataset is to partition it into training and test sets. A learning system is first trained on a set of images and their labels, then training is stopped and performance is measured on a separate set of test images from the same classes. To adapt ImageNet to continual learning while minimizing all other changes, we constructed a sequence of binary classification tasks by taking the classes in pairs. For example, the first task might be to distinguish cats from houses and the second might be to distinguish stop signs from school buses. With the 1,000 classes in our dataset, we were able to form half a million binary classification tasks in this way. For each task, a deep-learning network was first trained on a subset of the images for the two classes and then its performance was measured on a separate test set for the classes. After training and testing on one task, the next task began with a different pair of classes. We call this problem ‘Continual ImageNet’. In Continual ImageNet, the difficulty of tasks remains the same over time. A drop in performance would mean the network is losing its learning ability, a direct demonstration of loss of plasticity.

We applied a wide variety of standard deep-learning networks to Continual ImageNet and tested many learning algorithms and parameter settings. To assess the performance of the network on a task, we measured the percentage of test images that were correctly classified. The results shown in Fig. 1b are representative; they are for a feed-forward convolutional network and for a training procedure, using unmodified backpropagation, that performed well on this problem in the first few tasks.

a – c , In a sequence of binary classification tasks using ImageNet pictures ( a ), the conventional backpropagation algorithm loses plasticity at all step sizes ( b ), whereas the continual backpropagation, L2 regularization and Shrink and Perturb algorithms maintain plasticity, apparently indefinitely ( c ). All results are averaged over 30 runs; the solid lines represent the mean and the shaded regions correspond to ±1 standard error.

Although these networks learned up to 88% correct on the test set of the early tasks (Fig. 1b , left panel), by the 2,000th task, they had lost substantial plasticity for all values of the step-size parameter (right panel). Some step sizes performed well on the first two tasks but then much worse on subsequent tasks, eventually reaching a performance level below that of a linear network. For other step sizes, performance rose initially and then fell and was only slightly better than the linear network after 2,000 tasks. We found this to be a common pattern in our experiments: for a well-tuned network, performance first improves and then falls substantially, ending near or below the linear baseline. We have observed this pattern for many network architectures, parameter choices and optimizers. The specific choice of network architecture, algorithm parameters and optimizers affected when the performance started to drop, but a severe performance drop occurred for a wide range of choices. The failure of standard deep-learning methods to learn better than a linear network in later tasks is direct evidence that these methods do not work well in continual-learning problems.

Algorithms that explicitly keep the weights of the network small were an exception to the pattern of failure and were often able to maintain plasticity and even improve their performance over many tasks, as shown in Fig. 1c . L2 regularization adds a penalty for large weights; augmenting backpropagation with this enabled the network to continue improving its learning performance over at least 5,000 tasks. The Shrink and Perturb algorithm 11 , which includes L2 regularization, also performed well. Best of all was our continual backpropagation algorithm, which we discuss later. For all algorithms, we tested a wide range of parameter settings and performed many independent runs for statistical significance. The presented curves are the best representative of each algorithm.

For a second demonstration, we chose to use residual networks, class-incremental continual learning and the CIFAR-100 dataset. Residual networks include layer-skipping connections as well as the usual layer-to-layer connections of conventional convolutional networks. The residual networks of today are more widely used and produce better results than strictly layered networks 38 . Class-incremental continual learning 39 involves sequentially adding new classes while testing on all classes seen so far. In our demonstration, we started with training on five classes and then successively added more, five at a time, until all 100 were available. After each addition, the networks were trained and performance was measured on all available classes. We continued training on the old classes (unlike in most work in class-incremental learning) to focus on plasticity rather than on forgetting.

In this demonstration, we used an 18-layer residual network with a variable number of heads, adding heads as new classes were added. We also used further deep-learning techniques, including batch normalization, data augmentation, L2 regularization and learning-rate scheduling. These techniques are standardly used with residual networks and are necessary for good performance. We call this our base deep-learning system.

As more classes are added, correctly classifying images becomes more difficult and classification accuracy would decrease even if the network maintained its ability to learn. To factor out this effect, we compare the accuracy of our incrementally trained networks with networks that were retrained from scratch on the same subset of classes. For example, the network that was trained first on five classes, and then on all ten classes, is compared with a network retrained from scratch on all ten classes. If the incrementally trained network performs better than a network retrained from scratch, then there is a benefit owing to training on previous classes, and if it performs worse, then there is genuine loss of plasticity.

The red line in Fig. 2b shows that incremental training was initially better than retraining, but after 40 classes, the incrementally trained network showed loss of plasticity that became increasingly severe. By the end, when all 100 classes were available, the accuracy of the incrementally trained base system was 5% lower than the retrained network (a performance drop equivalent to that of removing a notable algorithmic advance, such as batch normalization). Loss of plasticity was less severe when Shrink and Perturb was added to the learning algorithm (in the incrementally trained network) and was eliminated altogether when continual backpropagation (see the ‘Maintaining plasticity through variability and selective preservation’ section) was added. These additions also prevented units of the network from becoming inactive or redundant, as shown in Fig. 2c,d .

a , An incrementally growing image-classification problem. b , Initially, accuracy is improved by incremental training compared with a network trained from scratch, but after 40 classes, accuracy degrades substantially in a base deep-learning system, less so for a Shrink and Perturb learning system and not at all for a learning system based on continual backpropagation. c , The number of network units that are active less than 1% of the time increases rapidly for the base deep-learning system, but less so for Shrink and Perturb and continual backpropagation systems. d , A low stable rank means that the units of a network do not provide much diversity; the base deep-learning system loses much more diversity than the Shrink and Perturb and continual backpropagation systems. All results are averaged over 30 runs; the solid lines represent the mean and the shaded regions correspond to ±1 standard error.

This demonstration involved larger networks and required more computation, but still we were able to perform extensive systematic tests. We found a robust pattern in the results that was similar to what we found in ImageNet. In both cases, deep-learning networks exhibited substantial loss of plasticity. Altogether, these results, along with other extensive results in Methods , constitute substantial evidence of plasticity loss.

Plasticity loss in reinforcement learning

Continual learning is essential to reinforcement learning in ways that go beyond its importance in supervised learning. Not only can the environment change but the behaviour of the learning agent can also change, thereby influencing the data it receives even if the environment is stationary. For this reason, the need for continual learning is often more apparent in reinforcement learning, and reinforcement learning is an important setting in which to demonstrate the tendency of deep learning towards loss of plasticity.

Nevertheless, it is challenging to demonstrate plasticity loss in reinforcement learning in a systematic and rigorous way. In part, this is because of the great variety of algorithms and experimental settings that are commonly used in reinforcement-learning research. Algorithms may learn value functions, behaviours or both simultaneously and may involve replay buffers, world models and learned latent states. Experiments may be episodic, continuing or offline. All of these choices involve several embedded choices of parameters. More fundamentally, reinforcement-learning algorithms affect the data seen by the agent. The learning ability of an algorithm is thus confounded with its ability to generate informative data. Finally, and in part because of the preceding, reinforcement-learning results tend to be more stochastic and more widely varying than in supervised learning. Altogether, demonstration of reinforcement-learning abilities, particularly negative results, tends to require more runs and generally much more experimental work and thus inevitably cannot be as definitive as in supervised learning.

Our first demonstration involves a reinforcement-learning algorithm applied to a simulated ant-like robot tasked with moving forwards as rapidly and efficiently as possible. The agent–environment interaction comprises a series of episodes, each beginning in a standard state and lasting up to 1,000 time steps. On each time step, the agent receives a reward depending on the forward distance travelled and the magnitude of its action (see Methods for details). An episode terminates in fewer than 1,000 steps if the ant jumps too high instead of moving forwards, as often happens early in learning. In the results to follow, we use the cumulative reward during an episode as our primary performance measure. To make the task non-stationary (and thereby emphasize plasticity), the coefficient of friction between the feet of the ant and the floor is changed after every 2 million time steps (but only at an episode boundary; details in Methods ). For fastest walking, the agent must adapt (relearn) its way of walking each time the friction changes. For this experiment, we used the proximal policy optimization (PPO) algorithm 40 . PPO is a standard deep reinforcement-learning algorithm based on backpropagation. It is widely used, for example, in robotics 9 , in playing real-time strategy games 41 and in aligning large language models from human feedback 42 .

PPO performed well (see the red line in Fig. 3c ) for the first 2 million steps, up until the first change in friction, but then performed worse and worse. Note how the performance of the other algorithms in Fig. 3c decreased each time the friction changed and then recovered as the agent adapted to the new friction, giving the plot a sawtooth appearance. PPO augmented with a specially tuned Adam optimizer 24 , 43 performed much better (orange line in Fig. 3c ) but still performed much worse over successive changes after the first two, indicating substantial loss of plasticity. On the other hand, PPO augmented with L2 regularization and continual backpropagation largely maintained their plasticity as the problem changed.

a , The reinforcement-learning agent controls torques at the eight joints of the simulated ant (red circles) to maximize forward motion and minimize penalties. b , Here we use a version of the ant problem in which the friction on contact with the ground is abruptly changed every 2 million time steps. c , The standard PPO learning algorithm fails catastrophically on the non-stationary ant problem. If the optimizer of PPO (Adam) is tuned in a custom way, then the failure is less severe, but adding continual backpropagation or L2 regularization is necessary to perform well indefinitely. These results are averaged over 100 runs; the solid lines represent the mean and the shaded regions represent the 95% bootstrapped confidence interval.

Now consider the same ant-locomotion task except with the coefficient of friction held constant at an intermediate value over 50 million time steps. The red line in Fig. 4a shows that the average performance of PPO increased for about 3 million steps but then collapsed. After 20 million steps, the ant is failing every episode and is unable to learn to move forwards efficiently. The red lines in the other panels of Fig. 4 provide further insight into the loss of plasticity of PPO. They suggest that the network may be losing plasticity in the same way as in our supervised learning results (see Fig. 2 and Extended Data Fig. 3c ). In both cases, most of the network’s units became dormant during the experiment, and the network markedly lost stable rank. The addition of L2 regularization mitigated the performance degradation by preventing continual growth of weights but also resulted in very small weights (Fig. 4d ), which prevented the agent from committing to good behaviour. The addition of continual backpropagation performed better overall. We present results for continual backpropagation only with (slight) L2 regularization, because without it, performance was highly sensitive to parameter settings. These results show that plasticity loss can be catastrophic in both deep reinforcement learning as well as deep supervised learning.

a , The four reinforcement-learning algorithms performed similarly on this and the non-stationary problem (compare with Fig. 3c ). b , c , A closer look inside the networks reveals a similar pattern as in supervised learning (compare with Fig. 2c,d ). d , The absolute values of the weights of the networks increased steadily under standard and tuned PPO, whereas they decreased and stayed small under L2 regularization with or without continual backpropagation. These results are averaged over 30 runs; the solid lines represent the mean and the shaded regions represent the 95% bootstrapped confidence interval.

Maintaining plasticity

Surprisingly, popular methods such as Adam, Dropout and normalization actually increased loss of plasticity (see Extended Data Fig. 4a ). L2 regularization, on the other hand, reduced loss of plasticity in many cases (purple line in Figs. 1 , 3 and 4 ). L2 regularization stops the weights from becoming too large by moving them towards zero at each step. The small weights allow the network to remain plastic. Another existing method that reduced loss of plasticity is Shrink and Perturb 11 (orange line in Figs. 1 and 2 ). Shrink and Perturb is L2 regularization plus small random changes in weights at each step. The injection of variability into the network can reduce dormancy and increase the diversity of the representation (Figs. 2 and 4 ). Our results indicate that non-growing weights and sustained variability in the network may be important for maintaining plasticity.

We now describe a variation of the backpropagation algorithm that is explicitly designed to inject variability into the network and keep some of its weights small. Conventional backpropagation has two main parts: initialization with small random weights before training and then gradient descent at each training step. The initialization provides variability initially, but, as we have seen, with continued training, variability tends to be lost, as well as plasticity along with it. To maintain the variability, our new algorithm, continual backpropagation, reinitializes a small number of units during training, typically fewer than one per step. To prevent disruption of what the network has already learned, only the least-used units are considered for reinitialization. See Methods for details.

The blue line in Fig. 1c shows the performance of continual backpropagation on Continual ImageNet. It mitigated loss of plasticity in Continual ImageNet while outperforming existing methods. Similarly, the blue lines in Fig. 2 show the performance of continual backpropagation on class-incremental CIFAR-100 and its effect on the evolution of dormant units and stable rank. Continual backpropagation fully overcame loss of plasticity, with a high stable rank and almost no dead units throughout learning.

In reinforcement learning, continual backpropagation was applied together with L2 regularization (a small amount of regularization was added to prevent excessive sensitivity to parameters in reinforcement-learning experiments). The blue line in Fig. 3 shows the performance of PPO with continual backpropagation on the ant-locomotion problem with changing friction. PPO with continual backpropagation performed much better than standard PPO, with little or no loss of plasticity. On the ant-locomotion problem with constant friction (Fig. 4 ), PPO with continual backpropagation continued improving throughout the experiment. The blue lines in Fig. 4b–d show the evolution of the correlates of loss of plasticity when we used continual backpropagation. PPO with continual backpropagation had few dormant units, a high stable rank and an almost constant average weight magnitude.

Our results are consistent with the idea that small weights reduce loss of plasticity and that a continual injection of variability further mitigates loss of plasticity. Although Shrink and Perturb adds variability to all weights, continual backpropagation does so selectively, and this seems to enable it to better maintain plasticity. Continual backpropagation involves a form of variation and selection in the space of neuron-like units, combined with continuing gradient descent. The variation and selection is reminiscent of trial-and-error processes in evolution and behaviour 44 , 45 , 46 , 47 and has precursors in many earlier ideas, including Keifer–Wolfowitz methods 48 and restart methods 49 in engineering and feature-search methods in machine learning 31 , 32 , 33 , 34 , 35 , 50 . Continual backpropagation brings a form of this old idea to modern deep learning. However, it is just one variation of this idea; other variations are possible and some of these have been explored in recent work 25 , 27 . We look forward to future work that explicitly compares and further refines these variations.

Deep learning is an effective and valuable technology in settings in which learning occurs in a special training phase and not thereafter. In settings in which learning must continue, however, we have shown that deep learning does not work. By deep learning, we mean the existing standard algorithms for learning in multilayer artificial neural networks and by not work, we mean that, over time, they fail to learn appreciably better than shallow networks. We have shown such loss of plasticity using supervised-learning datasets and reinforcement-learning tasks on which deep learning has previously excelled and for a wide range of networks and standard learning algorithms. Taking a closer look, we found that, during training, many of the networks’ neuron-like units become dormant, overcommitted and similar to each other, hampering the ability of the networks to learn new things. As they learn, standard deep-learning networks gradually and irreversibly lose their diversity and thus their ability to continue learning. Plasticity loss is often severe when learning continues for many tasks, but may not occur at all for small numbers of tasks.

The problem of plasticity loss is not intrinsic to deep learning. Deep artificial neural networks trained by gradient descent are perfectly capable of maintaining their plasticity, apparently indefinitely, as we have shown with the Shrink and Perturb algorithm and particularly with the new continual backpropagation algorithm. Both of these algorithms extend standard deep learning by adding a source of continuing variability to the weights of the network, and continual backpropagation restricts this variability to the units of the network that are at present least used, minimizing damage to the operation of the network. That is, continual backpropagation involves a form of variation and selection in the space of neuron-like units, combined with continuing gradient descent. This idea has many historical antecedents and will probably require further development to reach its most effective form.

Specifics of continual backpropagation

Continual backpropagation selectively reinitializes low-utility units in the network. Our utility measure, called the contribution utility, is defined for each connection or weight and each unit. The basic intuition behind the contribution utility is that the magnitude of the product of units’ activation and outgoing weight gives information about how valuable this connection is to its consumers. If the contribution of a hidden unit to its consumer is small, its contribution can be overwhelmed by contributions from other hidden units. In such a case, the hidden unit is not useful to its consumer. We define the contribution utility of a hidden unit as the sum of the utilities of all its outgoing connections. The contribution utility is measured as a running average of instantaneous contributions with a decay rate, η , which is set to 0.99 in all experiments. In a feed-forward neural network, the contribution utility, u l [ i ], of the i th hidden unit in layer l at time t is updated as

in which h l , i , t is the output of the i th hidden unit in layer l at time t , w l , i , k , t is the weight connecting the i th unit in layer l to the k th unit in layer l  + 1 at time t and n l +1 is the number of units in layer l  + 1.

When a hidden unit is reinitialized, its outgoing weights are initialized to zero. Initializing the outgoing weights as zero ensures that the newly added hidden units do not affect the already learned function. However, initializing the outgoing weight to zero makes the new unit vulnerable to immediate reinitialization, as it has zero utility. To protect new units from immediate reinitialization, they are protected from a reinitialization for maturity threshold m number of updates. We call a unit mature if its age is more than m . Every step, a fraction of mature units ρ , called the replacement rate, is reinitialized in every layer.

The replacement rate ρ is typically set to a very small value, meaning that only one unit is replaced after hundreds of updates. For example, in class-incremental CIFAR-100 (Fig. 2 ) we used continual backpropagation with a replacement rate of 10 −5 . The last layer of the network in that problem had 512 units. At each step, roughly 512 × 10 −5  = 0.00512 units are replaced. This corresponds to roughly one replacement after every 1/0.00512 ≈ 200 updates or one replacement after every eight epochs on the first five classes.

The final algorithm combines conventional backpropagation with selective reinitialization to continually inject random units from the initial distribution. Continual backpropagation performs a gradient descent and selective reinitialization step at each update. Algorithm 1 specifies continual backpropagation for a feed-forward neural network. In cases in which the learning system uses mini-batches, the instantaneous contribution utility can be used by averaging the utility over the mini-batch instead of keeping a running average to save computation (see Extended Data Fig. 5d for an example). Continual backpropagation overcomes the limitation of previous work 34 , 35 on selective reinitialization and makes it compatible with modern deep learning.

Algorithm 1

Continual backpropagation for a feed-forward network with L layers

Set replacement rate ρ , decay rate η and maturity threshold m

Initialize the weights w 0 ,…,  w L −1 , in which w l is sampled from distribution d l

Initialize utilities u 1 ,…,  u L −1 , number of units to replace c 1 ,…,  c L −1 , and ages a 1 ,…,  a L −1 to 0

For each input x t do

Forward pass: pass x t through the network to get the prediction \(\widehat{{{\bf{y}}}_{t}}\)

Evaluate: receive loss \(l({{\bf{x}}}_{t},\widehat{{{\bf{y}}}_{t}})\)

Backward pass: update the weights using SGD or one of its variants

For layer l in 1: L  − 1 do

Update age: a l  =  a l  + 1

Update unit utility: see equation ( 1 )

Find eligible units: n eligible  = number of units with age greater than m

Update number of units to replace: c l  =  c l  +  n eligible  ×  ρ

If c l  > 1

Find the unit with smallest utility and record its index as r

Reinitialize input weights: resample w l −1 [:, r ] from distribution d l

Reinitialize output weights: set w l [ r ,:] to 0

Reinitialize utility and age: set u l [ r ] = 0 and a l [ r ] = 0

Update number of units to replace: c l  =  c l  − 1

Details of Continual ImageNet

The ImageNet database we used consists of 1,000 classes, each of 700 images. The 700 images for each class were divided into 600 images for a training set and 100 images for a test set. On each binary classification task, the deep-learning network was first trained on the training set of 1,200 images and then its classification accuracy was measured on the test set of 200 images. The training consisted of several passes through the training set, called epochs. For each task, all learning algorithms performed 250 passes through the training set using mini-batches of size 100. All tasks used the downsampled 32 × 32 version of the ImageNet dataset, as is often done to save computation 51 .

All algorithms on Continual ImageNet used a convolutional network. The network had three convolutional-plus-max-pooling layers, followed by three fully connected layers, as detailed in Extended Data Table 3 . The final layer consisted of just two units, the heads, corresponding to the two classes. At task changes, the input weights of the heads were reset to zero. Resetting the heads in this way can be viewed as introducing new heads for the new tasks. This resetting of the output weights is not ideal for studying plasticity, as the learning system gets access to privileged information on the timing of task changes (and we do not use it in other experiments in this paper). We use it here because it is the standard practice in deep continual learning for this type of problem in which the learning system has to learn a sequence of independent tasks 52 .

In this problem, we reset the head of the network at the beginning of each task. It means that, for a linear network, the whole network is reset. That is why the performance of a linear network will not degrade in Continual ImageNet. As the linear network is a baseline, having a low-variance estimate of its performance is desirable. The value of this baseline is obtained by averaging over thousands of tasks. This averaging gives us a much better estimate of its performance than other networks.

The network was trained using SGD with momentum on the cross-entropy loss and initialized once before the first task. The momentum hyperparameter was 0.9. We tested various step-size parameters for backpropagation but only presented the performance for step sizes 0.01, 0.001 and 0.0001 for clarity of Fig. 1b . We performed 30 runs for each hyperparameter value, varying the sequence of tasks and other randomness. Across different hyperparameters and algorithms, the same sequences of pairs of classes were used.

We now describe the hyperparameter selection for L2 regularization, Shrink and Perturb and continual backpropagation. The main text presents the results for these algorithms on Continual ImageNet in Fig. 1c . We performed a grid search for all algorithms to find the set of hyperparameters that had the highest average classification accuracy over 5,000 tasks. The values of hyperparameters used for the grid search are described in Extended Data Table 2 . L2 regularization has two hyperparameters, step size and weight decay. Shrink and Perturb has three hyperparameters, step size, weight decay and noise variance. We swept over two hyperparameters of continual backpropagation: step size and replacement rate. The maturity threshold in continual backpropagation was set to 100. For both backpropagation and L2 regularization, the performance was poor for step sizes of 0.1 or 0.003. We chose to only use step sizes of 0.03 and 0.01 for continual backpropagation and Shrink and Perturb. We performed ten independent runs for all sets of hyperparameters. Then we performed another 20 runs to complete 30 runs for the best-performing set of hyperparameters to produce the results in Fig. 1c .

Class-incremental CIFAR-100

In the class-incremental CIFAR-100, the learning system gets access to more and more classes over time. Classes are provided to the learning system in increments of five. First, it has access to just five classes, then ten and so on, until it gets access to all 100 classes. The learning system is evaluated on the basis of how well it can discriminate between all the available classes at present. The dataset consists of 100 classes with 600 images each. The 600 images for each class were divided into 450 images to create a training set, 50 for a validation set and 100 for a test set. Note that the network is trained on all data from all classes available at present. First, it is trained on data from just five classes, then from all ten classes and so on, until finally, it is trained from data from all 100 classes simultaneously.

After each increment, the network was trained for 200 epochs, for a total of 4,000 epochs for all 20 increments. We used a learning-rate schedule that resets at the start of each increment. For the first 60 epochs of each increment, the learning rate was set to 0.1, then to 0.02 for the next 60 epochs, then 0.004 for the next 40 epochs and to 0.0008 for the last 40 epochs; we used the initial learning rate and learning-rate schedule reported in ref.  53 . During the 200 epochs of training for each increment, we kept track of the network with the best accuracy on the validation set. To prevent overfitting, at the start of each new increment, we reset the weights of the network to the weights of the best-performing (on the validation set) network found during the previous increment; this is equivalent to early stopping for each different increment.

We used an 18-layer deep residual network 38 for all experiments on class-incremental CIFAR-100. The network architecture is described in detail in Extended Data Table 1 . The weights of convolutional and linear layers were initialized using Kaiming initialization 54 , the weights for the batch-norm layers were initialized to one and all of the bias terms in the network were initialized to zero. Each time five new classes were made available to the network, five more output units were added to the final layer of the network. The weights and biases of these output units were initialized using the same initialization scheme. The weights of the network were optimized using SGD with a momentum of 0.9, a weight decay of 0.0005 and a mini-batch size of 90.

We used several steps of data preprocessing before the images were presented to the network. First, the value of all the pixels in each image was rescaled between 0 and 1 through division by 255. Then, each pixel in each channel was centred and rescaled by the average and standard deviation of the pixel values of each channel, respectively. Finally, we applied three random data transformations to each image before feeding it to the network: randomly horizontally flip the image with a probability of 0.5, randomly crop the image by padding the image with 4 pixels on each side and randomly cropping to the original size, and randomly rotate the image between 0 and 15°. The first two steps of preprocessing were applied to the training, validation and test sets, but the random transformations were only applied to the images in the training set.

We tested several hyperparameters to ensure the best performance for each different algorithm with our specific architecture. For the base system, we tested values for the weight decay parameter in {0.005, 0.0005, 0.00005}. A weight-decay value of 0.0005 resulted in the best performance in terms of area under the curve for accuracy on the test set over the 20 increments. For Shrink and Perturb, we used the weight-decay value of the base system and tested values for the standard deviation of the Gaussian noise in {10 −4 , 10 −5 , 10 −6 }; 10 −5 resulted in the best performance. For continual backpropagation, we tested values for the maturity threshold in {1,000, 10,000} and for the replacement rate in {10 −4 , 10 −5 , 10 −6 } using the contribution utility described in equation ( 1 ). A maturity threshold of 1,000 and a replacement rate of 10 −5 resulted in the best performance. Finally, for the head-resetting baseline, in Extended Data Fig. 1a , we used the same hyperparameters as for the base system, but the output layer was reinitialized at the start of each increment.

In Fig. 2d , we plot the stable rank of the representation in the penultimate layer of the network and the percentage of dead units in the full network. For a matrix \({\boldsymbol{\Phi }}\in {{\mathbb{R}}}^{n\times m}\) with singular values σ k sorted in descending order for k  = 1, 2,…,  q and q  = max( n ,  m ), the stable rank 55 is \(\min \left\{k:\frac{{\Sigma }_{i}^{k}{\sigma }_{i}}{{\Sigma }_{j}^{q}{\sigma }_{j}} > 0.99\right\}\) .

For reference, we also implemented a network with the same hyperparameters as the base system but that was reinitialized at the beginning of each increment. Figure 2b shows the performance of each algorithm relative to the performance of the reinitialized network. For completeness, Extended Data Fig. 1a shows the test accuracy of each algorithm in each different increment. The final accuracy of continual backpropagation on all 100 classes was 76.13%, whereas Extended Data Fig. 1b shows the performance of continual backpropagation for different replacement rates with a maturity threshold of 1,000. For all algorithms that we tested, there was no correlation between when a class was presented and the accuracy of that class, implying that the temporal order of classes did not affect performance.

Robust loss of plasticity in permuted MNIST

We now use a computationally cheap problem based on the MNIST dataset 56 to test the generality of loss of plasticity across various conditions. MNIST is one of the most common supervised-learning datasets used in deep learning. It consists of 60,000, 28 × 28, greyscale images of handwritten digits from 0 to 9, together with their correct labels. For example, the left image in Extended Data Fig. 3a shows an image that is labelled by the digit 7. The smaller number of classes and the simpler images enable much smaller networks to perform well on this dataset than are needed on ImageNet or CIFAR-100. The smaller networks in turn mean that much less computation is needed to perform the experiments and thus experiments can be performed in greater quantities and under a variety of different conditions, enabling us to perform deeper and more extensive studies of plasticity.

We created a continual supervised-learning problem using permuted MNIST datasets 57 , 58 . An individual permuted MNIST dataset is created by permuting the pixels in the original MNIST dataset. The right image in Extended Data Fig. 3a is an example of such a permuted image. Given a way of permuting, all 60,000 images are permuted in the same way to produce the new permuted MNIST dataset. Furthermore, we normalized pixel values between 0 and 1 by dividing by 255.

By repeatedly randomly selecting from the approximately 10 1930 possible permutations, we created a sequence of 800 permuted MNIST datasets and supervised-learning tasks. For each task, we presented each of its 60,000 images one by one in random order to the learning network. Then we moved to the next permuted MNIST task and repeated the whole procedure, and so on for up to 800 tasks. No indication was given to the network at the time of task switching. With the pixels being permuted in a completely unrelated way, we might expect classification performance to fall substantially at the time of each task switch. Nevertheless, across tasks, there could be some savings, some improvement in speed of learning or, alternatively, there could be loss of plasticity—loss of the ability to learn across tasks. The network was trained on a single pass through the data and there were no mini-batches. We call this problem Online Permuted MNIST.

We applied feed-forward neural networks with three hidden layers to Online Permuted MNIST. We did not use convolutional layers, as they could not be helpful on the permuted problem because the spatial information is lost; in MNIST, convolutional layers are often not used even on the standard, non-permuted problem. For each example, the network estimated the probabilities of each of the tem classes, compared them to the correct label and performed SGD on the cross-entropy loss. As a measure of online performance, we recorded the percentage of times the network correctly classified each of the 60,000 images in the task. We plot this per-task performance measure versus task number in Extended Data Fig. 3b . The weights were initialized according to a Kaiming distribution.

The left panel of Extended Data Fig. 3b shows the progression of online performance across tasks for a network with 2,000 units per layer and various values of the step-size parameter. Note that that performance first increased across tasks, then began falling steadily across all subsequent tasks. This drop in performance means that the network is slowly losing the ability to learn from new tasks. This loss of plasticity is consistent with the loss of plasticity that we observed in ImageNet and CIFAR-100.

Next, we varied the network size. Instead of 2,000 units per layer, we tried 100, 1,000 and 10,000 units per layer. We ran this experiment for only 150 tasks, primarily because the largest network took much longer to run. The performances at good step sizes for each network size are shown in the middle panel of Extended Data Fig. 3b . Loss of plasticity with continued training is most pronounced at the smaller network sizes, but even the largest networks show some loss of plasticity.

Next, we studied the effect of the rate at which the task changed. Going back to the original network with 2,000-unit layers, instead of changing the permutation after each 60,000 examples, we now changed it after each 10,000, 100,000 or 1 million examples and ran for 48 million examples in total no matter how often the task changed. The examples in these cases were selected randomly with replacement for each task. As a performance measure of the network on a task, we used the percentage correct over all of the images in the task. The progression of performance is shown in the right panel in Extended Data Fig. 3b . Again, performance fell across tasks, even if the change was very infrequent. Altogether, these results show that the phenomenon of loss of plasticity robustly arises in this form of backpropagation. Loss of plasticity happens for a wide range of step sizes, rates of distribution change and for both underparameterized and overparameterized networks.

Loss of plasticity with different activations in the Slowly-Changing Regression problem

There remains the issue of the network’s activation function. In our experiments so far, we have used ReLU, the most popular choice at present, but there are several other possibilities. For these experiments, we switched to an even smaller, more idealized problem. Slowly-Changing Regression is a computationally inexpensive problem in which we can run a single experiment on a CPU core in 15 min, allowing us to perform extensive studies. As its name suggests, this problem is a regression problem—meaning that the labels are real numbers, with a squared loss, rather than nominal values with a cross-entropy loss—and the non-stationarity is slow and continual rather than abrupt, as in a switch from one task to another. In Slowly-Changing Regression, we study loss of plasticity for networks with six popular activation functions: sigmoid, tanh, ELU 59 , leaky ReLU 60 , ReLU 61 and Swish 62 .

In Slowly-Changing Regression, the learner receives a sequence of examples. The input for each example is a binary vector of size m  + 1. The input has f slow-changing bits, m  −  f random bits and then one constant bit. The first f bits in the input vector change slowly. After every T examples, one of the first f bits is chosen uniformly at random and its value is flipped. These first f bits remain fixed for the next T examples. The parameter T allows us to control the rate at which the input distribution changes. The next m  −  f bits are randomly sampled for each example. Last, the ( m  + 1)th bit is a bias term with a constant value of one.

The target output is generated by running the input vector through a neural network, which is set at the start of the experiment and kept fixed. As this network generates the target output and represents the desired solution, we call it the target network. The weights of the target networks are randomly chosen to be +1 or −1. The target network has one hidden layer with the linear threshold unit (LTU) activation. The value of the i th LTU is one if the input is above a threshold θ i and 0 otherwise. The threshold θ i is set to be equal to ( m  + 1) ×  β  −  S i , in which β   ∈  [0, 1] and S i is the number of input weights with negative value 63 . The details of the input and target function in the Slowly-Changing Regression problem are also described in Extended Data Fig. 2a .

The details of the specific instance of the Slowly-Changing Regression problem we use in this paper and the learning network used to predict its output are listed in Extended Data Table 4 . Note that the target network is more complex than the learning network, as the target network is wider, with 100 hidden units, whereas the learner has just five hidden units. Thus, because the input distribution changes every T example and the target function is more complex than what the learner can represent, there is a need to track the best approximation.

We applied learning networks with different activation functions to the Slowly-Changing Regression. The learner used the backpropagation algorithm to learn the weights of the network. We used a uniform Kaiming distribution 54 to initialize the weights of the learning network. The distribution is U (− b ,  b ) with bound, \(b={\rm{g}}{\rm{a}}{\rm{i}}{\rm{n}}\times \sqrt{\frac{3}{{\rm{n}}{\rm{u}}{\rm{m}}{\rm{\_}}{\rm{i}}{\rm{n}}{\rm{p}}{\rm{u}}{\rm{t}}{\rm{s}}}}\) , in which gain is chosen such that the magnitude of inputs does not change across layers. For tanh, sigmoid, ReLU and leaky ReLU, the gain is 5/3, 1, \(\sqrt{2}\) and \(\sqrt{2/(1+{\alpha }^{2})}\) , respectively. For ELU and Swish, we used \({\rm{gain}}=\sqrt{2}\) , as was done in the original papers 59 , 62 .

We ran the experiment on the Slowly-Changing Regression problem for 3 million examples. For each activation and value of step size, we performed 100 independent runs. First, we generated 100 sequences of examples (input–output pairs) for the 100 runs. Then these 100 sequences of examples were used for experiments with all activations and values of the step-size parameter. We used the same sequence of examples to control the randomness in the data stream across activations and step sizes.

The results of the experiments are shown in Extended Data Fig. 2b . We measured the squared error, that is, the square of the difference between the true target and the prediction made by the learning network. In Extended Data Fig. 2b , the squared error is presented in bins of 40,000 examples. This means that the first data point is the average squared error on the first 40,000 examples, the next is the average squared error on the next 40,000 examples and so on. The shaded region in the figure shows the standard error of the binned error.

Extended Data Fig. 2b shows that, in Slowly-Changing Regression, after performing well initially, the error increases for all step sizes and activations. For some activations such as ReLU and tanh, loss of plasticity is severe, and the error increases to the level of the linear baseline. Although for other activations such as ELU loss of plasticity is less severe, there is still a notable loss of plasticity. These results mean that loss of plasticity is not resolved by using commonly used activations. The results in this section complement the results in the rest of the article and add to the generality of loss of plasticity in deep learning.

Understanding loss of plasticity

We now turn our attention to understanding why backpropagation loses plasticity in continual-learning problems. The only difference in the learner over time is the network weights. In the beginning, the weights were small random numbers, as they were sampled from the initial distribution; however, after learning some tasks, the weights became optimized for the most recent task. Thus, the starting weights for the next task are qualitatively different from those for the first task. As this difference in the weights is the only difference in the learning algorithm over time, the initial weight distribution must have some unique properties that make backpropagation plastic in the beginning. The initial random distribution might have many properties that enable plasticity, such as the diversity of units, non-saturated units, small weight magnitude etc.

As we now demonstrate, many advantages of the initial distribution are lost concurrently with loss of plasticity. The loss of each of these advantages partially explains the degradation in performance that we have observed. We then provide arguments for how the loss of these advantages could contribute to loss of plasticity and measures that quantify the prevalence of each phenomenon. We provide an in-depth study of the Online Permuted MNIST problem that will serve as motivation for several solution methods that could mitigate loss of plasticity.

The first noticeable phenomenon that occurs concurrently with the loss of plasticity is the continual increase in the fraction of constant units. When a unit becomes constant, the gradients flowing back from the unit become zero or very close to zero. Zero gradients mean that the weights coming into the unit do not change, which means that this unit loses all of its plasticity. In the case of ReLU activations, this occurs when the output of the activations is zero for all examples of the task; such units are often said to be dead 64 , 65 . In the case of the sigmoidal activation functions, this phenomenon occurs when the output of a unit is too close to either of the extreme values of the activation function; such units are often said to be saturated 66 , 67 .

To measure the number of dead units in a network with ReLU activation, we count the number of units with a value of zero for all examples in a random sample of 2,000 images at the beginning of each new task. An analogous measure in the case of sigmoidal activations is the number of units that are ϵ away from either of the extreme values of the function for some small positive ϵ (ref.  68 ). We only focus on ReLU networks in this section.

In our experiments on the Online Permuted MNIST problem, the deterioration of online performance is accompanied by a large increase in the number of dead units (left panel of Extended Data Fig. 3c ). For the step size of 0.01, up to 25% of units die after 800 tasks. In the permuted MNIST problem, in which all inputs are positive because they are normalized between 0 and 1, once a unit in the first layer dies, it stays dead forever. Thus, an increase in dead units directly decreases the total capacity of the network. In the next section, we will see that methods that stop the units from dying can substantially reduce loss of plasticity. This further supports the idea that the increase in dead units is one of the causes of loss of plasticity in backpropagation.

Another phenomenon that occurs with loss of plasticity is the steady growth of the network’s average weight magnitude. We measure the average magnitude of the weights by adding up their absolute values and dividing by the total number of weights in the network. In the permuted MNIST experiment, the degradation of online classification accuracy of backpropagation observed in Extended Data Fig. 3b is associated with an increase in the average magnitude of the weights (centre panel of Extended Data Fig. 3c ). The growth of the magnitude of the weights of the network can represent a problem because large weight magnitudes are often associated with slower learning. The weights of a neural network are directly linked to the condition number of the Hessian matrix in the second-order Taylor approximation of the loss function. The condition number of the Hessian is known to affect the speed of convergence of SGD algorithms (see ref.  69 for an illustration of this phenomenon in convex optimization). Consequently, the growth in the magnitude of the weights could lead to an ill-conditioned Hessian matrix, resulting in a slower convergence.

The last phenomenon that occurs with the loss of plasticity is the drop in the effective rank of the representation. Similar to the rank of a matrix, which represents the number of linearly independent dimensions, the effective rank takes into consideration how each dimension influences the transformation induced by a matrix 70 . A high effective rank indicates that most of the dimensions of the matrix contribute similarly to the transformation induced by the matrix. On the other hand, a low effective rank corresponds to most dimensions having no notable effect on the transformation, implying that the information in most of the dimensions is close to being redundant.

Formally, consider a matrix \(\Phi \in {{\mathbb{R}}}^{n\times m}\) with singular values σ k for k  = 1, 2,…,  q , and q  = max( n ,  m ). Let p k  =  σ k / ∥ σ ∥ 1 , in which σ is the vector containing all the singular values and ∥ ⋅ ∥ 1 is the ℓ 1 norm. The effective rank of matrix Φ , or erank( Φ ), is defined as

Note that the effective rank is a continuous measure that ranges between one and the rank of matrix Φ .

In the case of neural networks, the effective rank of a hidden layer measures the number of units that can produce the output of the layer. If a hidden layer has a low effective rank, then a small number of units can produce the output of the layer, meaning that many of the units in the hidden layer are not providing any useful information. We approximate the effective rank on a random sample of 2,000 examples before training on each task.

In our experiments, loss of plasticity is accompanied by a decrease in the average effective rank of the network (right panel of Extended Data Fig. 3c ). This phenomenon in itself is not necessarily a problem. After all, it has been shown that gradient-based optimization seems to favour low-rank solutions through implicit regularization of the loss function or implicit minimization of the rank itself 71 , 72 . However, a low-rank solution might be a bad starting point for learning from new observations because most of the hidden units provide little to no information.

The decrease in effective rank could explain the loss of plasticity in our experiments in the following way. After each task, the learning algorithm finds a low-rank solution for the current task, which then serves as the initialization for the next task. As the process continues, the effective rank of the representation layer keeps decreasing after each task, limiting the number of solutions that the network can represent immediately at the start of each new task.

In this section, we looked deeper at the networks that lost plasticity in the Online Permuted MNIST problem. We noted that the only difference in the learning algorithm over time is the weights of the network, which means that the initial weight distribution has some properties that allowed the learning algorithm to be plastic in the beginning. And as learning progressed, the weights of the network moved away from the initial distribution and the algorithm started to lose plasticity. We found that loss of plasticity is correlated with an increase in weight magnitude, a decrease in the effective rank of the representation and an increase in the fraction of dead units. Each of these correlates partially explains loss of plasticity faced by backpropagation.

Existing deep-learning methods for mitigating loss of plasticity

We now investigate several existing methods and test how they affect loss of plasticity. We study five existing methods: L2 regularization 73 , Dropout 74 , online normalization 75 , Shrink and Perturb 11 and Adam 43 . We chose L2 regularization, Dropout, normalization and Adam because these methods are commonly used in deep-learning practice. Although Shrink and Perturb is not a commonly used method, we chose it because it reduces the failure of pretraining, a problem that is an instance of loss of plasticity. To assess if these methods can mitigate loss of plasticity, we tested them on the Online Permuted MNIST problem using the same network architecture we used in the previous section, ‘Understanding loss of plasticity’. Similar to the previous section, we measure the online classification accuracy on all 60,000 examples of the task. All the algorithms used a step size of 0.003, which was the best-performing step size for backpropagation in the left panel of Extended Data Fig. 3b . We also use the three correlates of loss of plasticity found in the previous section to get a deeper understanding of the performance of these methods.

An intuitive way to address loss of plasticity is to use weight regularization, as loss of plasticity is associated with a growth of weight magnitudes, shown in the previous section. We used L2 regularization, which adds a penalty to the loss function proportional to the ℓ 2 norm of the weights of the network. The L2 regularization penalty incentivizes SGD to find solutions that have a low weight magnitude. This introduces a hyperparameter λ that modulates the contribution of the penalty term.

The purple line in the left panel of Extended Data Fig. 4a shows the performance of L2 regularization on the Online Permuted MNIST problem. The purple lines in the other panels of Extended Data Fig. 4a show the evolution of the three correlates of loss of plasticity with L2 regularization. For L2 regularization, the weight magnitude does not continually increase. Moreover, as expected, the non-increasing weight magnitude is associated with lower loss of plasticity. However, L2 regularization does not fully mitigate loss of plasticity. The other two correlates for loss of plasticity explain this, as the percentage of dead units kept increasing and the effective rank kept decreasing. Finally, Extended Data Fig. 4b shows the performance of L2 regularization for different values of λ . The regularization parameter λ controlled the peak of the performance and how quickly it decreased.

A method related to weight regularization is Shrink and Perturb 11 . As the name suggests, Shrink and Perturb performs two operations; it shrinks all the weights and then adds random Gaussian noise to these weights. The introduction of noise introduces another hyperparameter, the standard deviation of the noise. Owing to the shrinking part of Shrink and Perturb, the algorithm favours solutions with smaller average weight magnitude than backpropagation. Moreover, the added noise prevents units from dying because it adds a non-zero probability that a dead unit will become active again. If Shrink and Perturb mitigates these correlates to loss of plasticity, it could reduce loss of plasticity.

The performance of Shrink and Perturb is shown in orange in Extended Data Fig. 4 . Similar to L2 regularization, Shrink and Perturb stops the weight magnitude from continually increasing. Moreover, it also reduces the percentage of dead units. However, it has a lower effective rank than backpropagation, but still higher than that of L2 regularization. Not only does Shrink and Perturb have a lower loss of plasticity than backpropagation but it almost completely mitigates loss of plasticity in Online Permuted MNIST. However, Shrink and Perturb was sensitive to the standard deviation of the noise. If the noise was too high, loss of plasticity was much more severe, and if it was too low, it did not have any effect.

An important technique in modern deep learning is called Dropout 74 . Dropout randomly sets each hidden unit to zero with a small probability, which is a hyperparameter of the algorithm. The performance of Dropout is shown in pink in Extended Data Fig. 4 .

Dropout showed similar measures of percentage of dead units, weight magnitude and effective rank as backpropagation, but, surprisingly, showed higher loss of plasticity. The poor performance of Dropout is not explained by our three correlates of loss of plasticity, which means that there are other possible causes of loss of plasticity. A thorough investigation of Dropout is beyond the scope of this paper, though it would be an interesting direction for future work. We found that a higher Dropout probability corresponded to a faster and sharper drop in performance. Dropout with probability of 0.03 performed the best and its performance was almost identical to that of backpropagation. However, Extended Data Fig. 4a shows the performance for a Dropout probability of 0.1 because it is more representative of the values used in practice.

Another commonly used technique in deep learning is batch normalization 76 . In batch normalization, the output of each hidden layer is normalized and rescaled using statistics computed from each mini-batch of data. We decided to include batch normalization in this investigation because it is a popular technique often used in practice. Because batch normalization is not amenable to the online setting used in the Online Permuted MNIST problem, we used online normalization 77 instead, an online variant of batch normalization. Online normalization introduces two hyperparameters used for the incremental estimation of the statistics in the normalization steps.

The performance of online normalization is shown in green in Extended Data Fig. 4 . Online normalization had fewer dead units and a higher effective rank than backpropagation in the earlier tasks, but both measures deteriorated over time. In the later tasks, the network trained using online normalization has a higher percentage of dead units and a lower effective rank than the network trained using backpropagation. The online classification accuracy is consistent with these results. Initially, it has better classification accuracy, but later, its classification accuracy becomes lower than that of backpropagation. For online normalization, the hyperparameters changed when the performance of the method peaked, and it also slightly changed how fast it got to its peak performance.

No assessment of alternative methods can be complete without Adam 43 , as it is considered one of the most useful tools in modern deep learning. The Adam optimizer is a variant of SGD that uses an estimate of the first moment of the gradient scaled inversely by an estimate of the second moment of the gradient to update the weights instead of directly using the gradient. Because of its widespread use and success in both supervised and reinforcement learning, we decided to include Adam in this investigation to see how it would affect the plasticity of deep neural networks. Adam has two hyperparameters that are used for computing the moving averages of the first and second moments of the gradient. We used the default values of these hyperparameters proposed in the original paper and tuned the step-size parameter.

The performance of Adam is shown in cyan in Extended Data Fig. 4 . Adam’s loss of plasticity can be categorized as catastrophic, as it reduces substantially. Consistent with our previous results, Adam scores poorly in the three measures corresponding to the correlates of loss of plasticity. Adam had an early increase in the percentage of dead units that plateaus at around 60%, similar weight magnitude as backpropagation and a large drop in the effective rank early during training. We also tested Adam with different activation functions on the Slowly-Changing Regression and found that loss of plasticity with Adam is usually worse than with SGD.

Many of the standard methods substantially worsened loss of plasticity. The effect of Adam on the plasticity of the networks was particularly notable. Networks trained with Adam quickly lost almost all of their diversity, as measured by the effective rank, and gained a large percentage of dead units. This marked loss of plasticity of Adam is an important result for deep reinforcement learning, for which Adam is the default optimizer 78 , and reinforcement learning is inherently continual owing to the ever-changing policy. Similar to Adam, other commonly used methods such as Dropout and normalization worsened loss of plasticity. Normalization had better performance in the beginning, but later it had a sharper drop in performance than backpropagation. In the experiment, Dropout simply made the performance worse. We saw that the higher the Dropout probability, the larger the loss of plasticity. These results mean that some of the most successful tools in deep learning do not work well in continual learning, and we need to focus on directly developing tools for continual learning.

We did find some success in maintaining plasticity in deep neural networks. L2 regularization and Shrink and Perturb reduce loss of plasticity. Shrink and Perturb is particularly effective, as it almost entirely mitigates loss of plasticity. However, both Shrink and Perturb and L2 regularization are slightly sensitive to hyperparameter values. Both methods only reduce loss of plasticity for a small range of hyperparameters, whereas for other hyperparameter values, they make loss of plasticity worse. This sensitivity to hyperparameters can limit the application of these methods to continual learning. Furthermore, Shrink and Perturb does not fully resolve the three correlates of loss of plasticity, it has a lower effective rank than backpropagation and it still has a high fraction of dead units.

We also applied continual backpropagation on Online Permuted MNIST. The replacement rate is the main hyperparameter in continual backpropagation, as it controls how rapidly units are reinitialized in the network. For example, a replacement rate of 10 −6 for our network with 2,000 hidden units in each layer would mean replacing one unit in each layer after every 500 examples.

Blue lines in Extended Data Fig. 4 show the performance of continual backpropagation. It has a non-degrading performance and is stable for a wide range of replacement rates. Continual backpropagation also mitigates all three correlates of loss of plasticity. It has almost no dead units, stops the network weights from growing and maintains a high effective rank across tasks. All algorithms that maintain a low weight magnitude also reduced loss of plasticity. This supports our claim that low weight magnitudes are important for maintaining plasticity. The algorithms that maintain low weight magnitudes were continual backpropagation, L2 regularization and Shrink and Perturb. Shrink and Perturb and continual backpropagation have an extra advantage over L2 regularization: they inject randomness into the network. This injection of randomness leads to a higher effective rank and lower number of dead units, which leads to both of these algorithms performing better than L2 regularization. However, continual backpropagation injects randomness selectively, effectively removing all dead units from the network and leading to a higher effective rank. This smaller number of dead units and a higher effective rank explains the better performance of continual backpropagation.

Details and further analysis in reinforcement learning

The experiments presented in the main text were conducted using the Ant-v3 environment from OpenAI Gym 79 . We changed the coefficient of friction by sampling it log-uniformly from the range [0.02, 2.00], using a logarithm with base 10. The coefficient of friction changed at the first episode boundary after 2 million time steps had passed since the last change. We also tested Shrink and Perturb on this problem and found that it did not provide a marked performance improvement over L2 regularization. Two separate networks were used for the policy and the value function, and both had two hidden layers with 256 units. These networks were trained using Adam alongside PPO to update the weights in the network. See Extended Data Table 5 for the values of the other hyperparameters. In all of the plots showing results of reinforcement-learning experiments, the shaded region represents the 95% bootstrapped confidence 80 .

The reward signal in the ant problem consists of four components. The main component rewards the agent for forward movement. It is proportional to the distance moved by the ant in the positive x direction since the last time step. The second component has a value of 1 at each time step. The third component penalizes the ant for taking large actions. This component is proportional to the square of the magnitude of the action. Finally, the last component penalizes the agent for large external contact forces. It is proportional to the sum of external forces (clipped in a range). The reward signal at each time step is the sum of these four components.

We also evaluated PPO and its variants in two more environments: Hopper-v3 and Walker-v3. The results for these experiments are presented in Extended Data Fig. 5a . The results mirrored those from Ant-v3; standard PPO suffered from a notable degradation in performance, in which its performance decreased substantially. However, this time, L2 regularization did not fix the issue in all cases; there was some performance degradation with L2 in Walker-v3. PPO, with continual backpropagation and L2 regularization, completely fixed the issue in all environments. Note that the only difference between our experiments and what is typically done in the literature is that we run the experiments for longer. Typically, these experiments are only done for 3 million steps, but we ran these experiments for up to 100 million steps.

PPO with L2 regularization only avoided degradation for a relatively large value of weight decay, 10 −3 . This extreme regularization stops the agent from finding better policies and stays stuck at a suboptimal policy. There was large performance degradation for smaller values of weight decay, and for larger values, the performance was always low. When we used continual backpropagation and L2 regularization together, we could use smaller values of weight decay. All the results for PPO with continual backpropagation and L2 regularization have a weight decay of 10 −4 , a replacement rate of 10 −4 and a maturity threshold of 10 4 . We found that the performance of PPO with continual backpropagation and L2 regularization was sensitive to the replacement rate but not to the maturity threshold and weight decay.

PPO uses the Adam optimizer, which keeps running estimates of the gradient and the squared of the gradient. These estimates require two further parameters, called β 1 and β 2 . The standard values of β 1 and β 2 are 0.9 and 0.999, respectively, which we refer to as standard Adam. Lyle et al. 24 showed that the standard values of β 1 and β 2 cause a large loss of plasticity. This happens because of the mismatch in β 1 and β 2 . A sudden large gradient can cause a very large update, as a large value of β 2 means that the running estimate for the square of the gradient, which is used in the denominator, is updated much more slowly than the running estimate for the gradient, which is the numerator. This loss of plasticity in Adam can be reduced by setting β 1 equal to β 2 . In our experiments, we set β 1 and β 2 to 0.99 and refer to it as tuned Adam/PPO. In Extended Data Fig. 5c , we measure the largest total weight change in the network during a single update cycle for bins of 1 million steps. The first point in the plots shows the largest weight change in the first 1 million steps. The second point shows the largest weight change in the second 1 second steps and so on. The figure shows that standard Adam consistently causes very large updates to the weights, which can destabilize learning, whereas tuned Adam with β 1  =  β 2  = 0.99 has substantially smaller updates, which leads to more stable learning. In all of our experiments, all algorithms other than the standard PPO used the tuned parameters for Adam ( β 1  =  β 2  = 0.99). The failure of standard Adam with PPO is similar to the failure of standard Adam in permuted MNIST.

In our next experiment, we perform a preliminary comparison with ReDo 25 . ReDo is another selective reinitialization method that builds on continual backpropagation but uses a different measure of utility and strategy for reinitializing. We tested ReDo on Ant-v3, the hardest of the three environments. ReDo requires two parameters: a threshold and a reinitialization period. We tested ReDo for all combinations of thresholds in {0.01, 0.03, 0.1} and reinitialization periods in {10, 10 2 , 10 3 , 10 4 , 10 5 }; a threshold of 0.1 with a reinitialization period of 10 2 performed the best. The performance of PPO with ReDo is plotted in Extended Data Fig. 5b . ReDo and continual backpropagation were used with weight decay of 10 −4 and β 1 and β 2 of 0.99. The figure shows that PPO with ReDo and L2 regularization performs much better than standard PPO. However, it still suffers from performance degradation and its performance is worse than PPO with L2 regularization. Note that this is only a preliminary comparison; we leave a full comparison and analysis of both methods for future work.

The performance drop of PPO in stationary environments is a nuanced phenomenon. Loss of plasticity and forgetting are both responsible for the observed degradation in performance. The degradation in performance implies that the agent forgot the good policy it had once learned, whereas the inability of the agent to relearn a good policy means it lost plasticity.

Loss of plasticity expresses itself in various forms in deep reinforcement learning. Some work found that deep reinforcement learning systems can lose their generalization abilities in the presence of non-stationarities 81 . A reduction in the effective rank, similar to the rank reduction in CIFAR-100, has been observed in some deep reinforcement-learning algorithms 82 . Nikishin et al. 18 showed that many reinforcement-learning systems perform better if their network is occasionally reset to its naive initial state, retaining only the replay buffer. This is because the learning networks became worse than a reinitialized network at learning from new data. Recent work has improved performance in many reinforcement-learning problems by applying plasticity-preserving methods 25 , 83 , 84 , 85 , 86 , 87 . These works focused on deep reinforcement learning systems that use large replay buffers. Our work complements this line of research as we studied systems based on PPO, which has much smaller replay buffers. Loss of plasticity is most relevant for systems that use small or no replay buffers, as large buffers can hide the effect of new data. Overcoming loss of plasticity is an important step towards deep reinforcement-learning systems that can learn from an online data stream.

Extended discussion

There are two main goals in continual learning: maintaining stability and maintaining plasticity 88 , 89 , 90 , 91 . Maintaining stability is concerned with memorizing useful information and maintaining plasticity is about finding new useful information when the data distribution changes. Current deep-learning methods struggle to maintain stability as they tend to forget previously learned information 28 , 29 . Many papers have been dedicated to maintaining stability in deep continual learning 30 , 92 , 93 , 94 , 95 , 96 , 97 . We focused on continually finding useful information, not on remembering useful information. Our work on loss of plasticity is different but complementary to the work on maintaining stability. Continual backpropagation in its current form does not tackle the forgetting problem. Its current utility measure only considers the importance of units for current data. One idea to tackle forgetting is to use a long-term measure of utility that remembers which units were useful in the past. Developing methods that maintain both stability and plasticity is an important direction for future work.

There are many desirable properties for an efficient continual-learning system 98 , 99 . It should be able to keep learning new things, control what it remembers and forgets, have good computational and memory efficiency and use previous knowledge to speed up learning on new data. The choice of the benchmark affects which property is being focused on. Most benchmarks and evaluations in our paper only focused on plasticity but not on other aspects, such as forgetting and speed of learning. For example, in Continual ImageNet, previous tasks are rarely repeated, which makes it effective for studying plasticity but not forgetting. In permuted MNIST, consecutive tasks are largely independent, which makes it suitable for studying plasticity in isolation. However, this independence means that previous knowledge cannot substantially speed up learning on new tasks. On the other hand, in class-incremental CIFAR-100, previous knowledge can substantially speed up learning of new classes. Overcoming loss of plasticity is an important, but still the first, step towards the goal of fast learning on future data 100 , 101 , 102 . Once we have networks that maintain plasticity, we can develop methods that use previous knowledge to speed up learning on future data.

Loss of plasticity is a critical factor when learning continues for many tasks, but it might be less important if learning happens for a small number of tasks. Usually, the learning system can take advantage of previous learning in the first few tasks. For example, in class-incremental CIFAR-100 (Fig. 2 ), the base deep-learning systems performed better than the network trained from scratch for up to 40 classes. This result is consistent with deep-learning applications in which the learning system is first trained on a large dataset and then fine-tuned on a smaller, more relevant dataset. Plasticity-preserving methods such as continual backpropagation may still improve performance in such applications based on fine-turning, but we do not expect that improvement to be large, as learning happens only for a small number of tasks. We have observed that deep-learning systems gradually lose plasticity, and this effect accumulates over tasks. Loss of plasticity becomes an important factor when learning continues for a large number of tasks; in class-incremental CIFAR-100, the performance of the base deep-learning system was much worse after 100 classes.

We have made notable progress in understanding loss of plasticity. However, it remains unclear which specific properties of initialization with small random numbers are important for maintaining plasticity. Recent work 103 , 104 has made exciting progress in this direction and it remains an important avenue for future work. The type of loss of plasticity studied in this article is largely because of the loss of the ability to optimize new objectives. This is different from the type of loss of plasticity in which the system can keep optimizing new objectives but lose the ability to generalize 11 , 12 . However, it is unclear if the two types of plasticity loss are fundamentally different or if the same mechanism can explain both phenomena. Future work that improves our understanding of plasticity and finds the underlying causes of both types of plasticity loss will be valuable to the community.

Continual backpropagation uses a utility measure to find and replace low-utility units. One limitation of continual backpropagation is that the utility measure is based on heuristics. Although it performs well, future work on more principled utility measures will improve the foundations of continual backpropagation. Our current utility measure is not a global measure of utility as it does not consider how a given unit affects the overall represented function. One possibility is to develop utility measures in which utility is propagated backwards from the loss function. The idea of utility in continual backpropagation is closely related to connection utility in the neural-network-pruning literature. Various papers 105 , 106 , 107 , 108 have proposed different measures of connection utility for the network-pruning problem. Adapting these utility measures to mitigate loss of plasticity is a promising direction for new algorithms and some recent work is already making progress in this direction 109 .

The idea of selective reinitialization is similar to the emerging idea of dynamic sparse training 110 , 111 , 112 . In dynamic sparse training, a sparse network is trained from scratch and connections between different units are generated and removed during training. Removing connections requires a measure of utility, and the initialization of new connections requires a generator similar to selective reinitialization. The main difference between dynamic sparse training and continual backpropagation is that dynamic sparse training operates on connections between units, whereas continual backpropagation operates on units. Consequently, the generator in dynamic sparse training must also decide which new connections to grow. Dynamic sparse training has achieved promising results in supervised and reinforcement-learning problems 113 , 114 , 115 , in which dynamic sparse networks achieve performance close to dense networks even at high sparsity levels. Dynamic sparse training is a promising idea that can be useful to maintain plasticity.

The idea of adding new units to neural networks is present in the continual-learning literature 92 , 116 , 117 . This idea is usually manifested in algorithms that dynamically increase the size of the network. For example, one method 117 expands the network by allocating a new subnetwork whenever there is a new task. These methods do not have an upper limit on memory requirements. Although these methods are related to the ideas in continual backpropagation, none are suitable for comparison, as continual backpropagation is designed for learning systems with finite memory, which are well suited for lifelong learning. And these methods would therefore require non-trivial modification to apply to our setting of finite memory.

Previous works on the importance of initialization have focused on finding the correct weight magnitude to initialize the weights. It has been shown that it is essential to initialize the weights so that the gradients do not become exponentially small in the initial layers of a network and the gradient is preserved across layers 54 , 66 . Furthermore, initialization with small weights is critical for sigmoid activations as they may saturate if the weights are too large 118 . Despite all this work on the importance of initialization, the fact that its benefits are only present initially but not continually has been overlooked, as these papers focused on cases in which learning has to be done just once, not continually.

Continual backpropagation selectively reinitializes low-utility units. One common strategy to deal with non-stationary data streams is reinitializing the network entirely. In the Online Permuted MNIST experiment, full reinitialization corresponds to a performance that stays at the level of the first point (Extended Data Fig. 4a ). In this case, continual backpropagation outperforms full reinitialization as it takes advantage of what it has previously learned to speed up learning on new data. In ImageNet experiments, the final performance of continual backpropagation is only slightly better than a fully reinitialized network (the first point for backpropagation in left panel of Fig. 1b ). However, Fig. 1 does not show how fast an algorithm reaches the final performance in each task. We observed that continual backpropagation achieves the best accuracy ten times faster than a fully reinitialized network on the 5,000th task of Continual ImageNet, ten epochs versus about 125 epochs. Furthermore, continual backpropagation could be combined with other methods that mitigate forgetting, which can further speed up learning on new data. In reinforcement learning, full reinitialization is only practical for systems with a large buffer. For systems that keep a small or no buffer, such as those we studied, full reinitialization will lead the agent to forget everything it has learned, and its performance will be down to the starting point.

Loss of plasticity might also be connected to the lottery ticket hypothesis 119 . The hypothesis states that randomly initialized networks contain subnetworks that can achieve performance close to that of the original network with a similar number of updates. These subnetworks are called winning tickets. We found that, in continual-learning problems, the effective rank of the representation at the beginning of tasks reduces over time. In a sense, the network obtained after training on several tasks has less randomness and diversity than the original random network. The reduced randomness might mean that the network has fewer winning tickets. And this reduced number of winning tickets might explain loss of plasticity. Our understanding of loss of plasticity could be deepened by fully exploring its connection with the lottery ticket hypothesis.

Some recent works have focused on quickly adapting to the changes in the data stream 120 , 121 , 122 . However, the problem settings in these papers were offline as they had two separate phases, one for learning and the other for evaluation. To use these methods online, they have to be pretrained on tasks that represent tasks that the learner will encounter during the online evaluation phase. This requirement of having access to representative tasks in the pretraining phase is not realistic for lifelong learning systems as the real world is non-stationary, and even the distribution of tasks can change over time. These methods are not comparable with those we studied in our work, as we studied fully online methods that do not require pretraining.

In this work, we found that methods that continually injected randomness while maintaining small weight magnitudes greatly reduced loss of plasticity. Many works have found that adding noise while training neural networks can improve training and testing performance. The main benefits of adding noise have been reported to be avoiding overfitting and improving training performance 123 , 124 , 125 . However, it can be tricky to inject noise without degrading performance in some cases 126 . In our case, when the data distribution is non-stationary, we found that continually injecting noise along with L2 regularization helps with maintaining plasticity in neural networks.

Data availability

All of the datasets and simulation environments used in this work are publicly available. Other data needed to evaluate the conclusions in the article are present in the article or the extended data.

Code availability

The code is available at https://github.com/shibhansh/loss-of-plasticity .

Rumelhart, D. E., Hinton, G. E. & Williams, R. J. Learning representations by back-propagating errors. Nature 323 , 533–536 (1986).

Article   ADS   Google Scholar  

OpenAI et al. GPT-4 technical report. Preprint at https://arxiv.org/abs/2303.08774 (2023).

Bahdanau, D., Cho, K. & Bengio, Y. Neural machine translation by jointly learning to align and translate. In Proc. 3rd International Conference on Learning Representations (eds Bengio, Y. & LeCun, Y.) (ICLR, 2015).

Jumper, J. et al. Highly accurate protein structure prediction with AlphaFold. Nature 596 , 583–589 (2021).

Article   ADS   CAS   PubMed   PubMed Central   Google Scholar  

Mnih, V. et al. Human-level control through deep reinforcement learning. Nature 518 , 529–533 (2015).

Article   ADS   CAS   PubMed   Google Scholar  

Silver, D. et al. Mastering the game of Go with deep neural networks and tree search. Nature 529 , 484–489 (2016).

Moravčík, M. et al. DeepStack: expert-level artificial intelligence in heads-up no-limit poker. Science 356 , 508–513 (2017).

Article   ADS   MathSciNet   PubMed   Google Scholar  

Wurman, P. R. et al. Outracing champion Gran Turismo drivers with deep reinforcement learning. Nature 602 , 223–228 (2022).

Andrychowicz, O. M. et al. Learning dexterous in-hand manipulation. Int. J. Robot. Res. 39 , 3–20 (2020).

Article   Google Scholar  

Kaufmann, E. et al. Champion-level drone racing using deep reinforcement learning. Nature 620 , 982–987 (2023).

Ash, J. & Adams, R. P. On warm-starting neural network training. Adv. Neural Inf. Process. Syst. 33 , 3884–3894 (2020).

Google Scholar  

Berariu, T. et al. A study on the plasticity of neural networks. Preprint at https://arxiv.org/abs/2106.00042 (2021).

Ellis, A. W. & Lambon Ralph, M. A. Age of acquisition effects in adult lexical processing reflect loss of plasticity in maturing systems: insights from connectionist networks. J. Exp. Psychol. Learn. Mem. Cogn. 26 , 1103 (2000).

Article   CAS   PubMed   Google Scholar  

Zevin, J. D. & Seidenberg, M. S. Age of acquisition effects in word reading and other tasks. J. Mem. Lang. 47 , 1–29 (2002).

Bonin, P., Barry, C., Méot, A. & Chalard, M. The influence of age of acquisition in word reading and other tasks: a never ending story? J. Mem. Lang. 50 , 456–476 (2004).

Chaudhry, A., Dokania, P. K., Ajanthan, T. & Torr, P. H. Riemannian walk for incremental learning: understanding forgetting and intransigence. In Proc. 15th European Conference on Computer Vision (ECCV) 532–547 (Springer, 2018).

Achille, A., Rovere, M. & Soatto, S. Critical learning periods in deep networks. In Proc. 6th International Conference on Learning Representations (eds Murray, I., Ranzato, M. & Vinyals, O.) (ICLR, 2018).

Nikishin, E., Schwarzer, M., D’Oro, P., Bacon, P.-L. & Courville, A. The primacy bias in deep reinforcement learning. In Proc. 39th International Conference on Machine Learning 16828–16847 (PMLR, 2022).

Dohare, S. The Interplay of Search and Gradient Descent in Semi-stationary Learning Problems . Master’s thesis, Univ. Alberta (2020).

Rahman, P. Toward Generate-and-test Algorithms for Continual Feature Discovery . Master’s thesis, Univ. Alberta (2021).

Dohare, S., Sutton, R. S. & Mahmood, A. R. Continual backprop: stochastic gradient descent with persistent randomness. Preprint at https://arxiv.org/abs/2108.06325 (2021).

Lyle, C., Rowland, M. & Dabney, W. Understanding and preventing capacity loss in reinforcement learning. In Proc. 10th International Conference on Learning Representations (ICLR, 2022).

Abbas, Z., Zhao, R., Modayil, J., White, A. & Machado, M. C. Loss of plasticity in continual deep reinforcement learning. In Proc. 2nd Conference on Lifelong Learning Agents (PMLR, 2023).

Lyle, C. et al. Understanding plasticity in neural networks. In Proc. 40th International Conference on Machine Learning 23190–23211 (PMLR, 2023).

Sokar, G., Agarwal, R., Castro, P. S. & Evci, U. The dormant neuron phenomenon in deep reinforcement learning. In Proc. 40th International Conference on Machine Learning 32145–32168 (PMLR, 2023).

Dohare, S., Hernandez-Garcia, J. F., Rahman, P., Mahmood, A. R. & Sutton, R. S. Maintaining plasticity in deep continual learning. Preprint at https://arxiv.org/abs/2306.13812 (2023).

Kumar, S., Marklund, H. & Van Roy, B. Maintaining plasticity in continual learning via regenerative regularization. In Proc. 3rd Conference on Lifelong Learning Agents (PMLR, 2024).

McCloskey, M. & Cohen, N. J. Catastrophic interference in connectionist networks: the sequential learning problem. Psychol. Learn. Motiv. 24 , 109–165 (1989).

French, R. M. Catastrophic forgetting in connectionist networks. Trends Cogn. Sci. 3 , 128–135 (1999).

Kirkpatrick, J. et al. Overcoming catastrophic forgetting in neural networks. Proc. Natl Acad. Sci. 114 , 3521–3526 (2017).

Article   ADS   MathSciNet   CAS   PubMed   Google Scholar  

Selfridge, O. G. Pandemonium: a paradigm for learning. In Mechanization of Thought Processes: Proceedings of a Symposium Held at the National Physical Laboratory 511–531 (Her Majesty’s Stationary Office, 1958).

Klopf, A. H. & Gose, E. An evolutionary pattern recognition network. IEEE Trans. Syst. Sci. Cybern. 5 , 247–250 (1969).

Holland, J. H. & Reitman, J. S. Cognitive systems based on adaptive algorithms. ACM SIGART Bull. 63 , 49–49 (1977).

Kaelbling, L. P. Learning in Embedded Systems (MIT Press, 1993).

Mahmood, A. R. & Sutton, R. S. Representation search through generate and test. In Proc. AAAI Workshop: Learning Rich Representations from Low-Level Sensors 16–21 (2013).

Deng, J. et al. ImageNet: a large-scale hierarchical image database. In Proc. 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 248–255 (IEEE, 2009).

Krizhevsky, A., Sutskever, I. & Hinton, G. E. ImageNet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 25 , 1097–1105 (2012).

He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 770–778 (2016).

Rebuffi, S.-A., Kolesnikov, A., Sperl, G. & Lampert, C. H. iCaRL: incremental classifier and representation learning. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2001–2010 (2017).

Schulman, J., Wolski, F., Dhariwal, P., Radford, A. & Klimov, O. Proximal policy optimization algorithms. Preprint at https://arxiv.org/abs/1707.06347 (2017).

OpenAI et al. Dota 2 with large scale deep reinforcement learning. Preprint at https://arxiv.org/abs/1912.06680 (2019).

Ouyang, L. et al. Training language models to follow instructions with human feedback. Adv. Neural Inf. Process. Syst. 36 , 27730–27744 (2022).

Kingma, D. P. & Ba, J. Adam: a method for stochastic optimization. In Proc. 3rd International Conference on Learning Representations (eds Bengio, Y. & LeCun, Y.) (ICLR, 2015).

Campbell, D. T. Blind variation and selective survival as a general strategy in knowledge-processes. Psychol. Rev. 67 , 380–400 (1960).

Thorndike, E. L. Animal Intelligence (Macmillan, 1911).

Dennett, D. C. Why the law of effect will not go away. J. Theory Soc. Behav. 5 , 169–187 (1975).

Holland, J. H. Adaptation in Natural and Artificial Systems (MIT Press, 1992).

Kashyap, R., Blaydon, C. & Fu, K. in Adaptive, Learning and Pattern Recognition Systems: Theory and Applications (eds Mendel, J. & Fu, K.) 329–355 (Elsevier, 1970).

Powell, M. J. D. Restart procedures for the conjugate gradient method. Math. Program. 12 , 241–254 (1977).

Article   MathSciNet   Google Scholar  

Stanley, K. O. & Miikkulainen, R. Evolving neural networks through augmenting topologies. Evol. Comput. 10 , 99–127 (2002).

Article   PubMed   Google Scholar  

Chrabaszcz, P., Loshchilov, I. & Hutter, F. A downsampled variant of ImageNet as an alternative to the CIFAR datasets. Preprint at https://arxiv.org/abs/1707.08819 (2017).

van de Ven, G. M., Tuytelaars, T. & Tolias, A. S. Three types of incremental learning. Nat. Mach. Intell. 4 , 1185–1197 (2022).

Weiaicunzai. pytorch-cifar100. GitHub https://github.com/weiaicunzai/pytorch-cifar100 (2022).

He, K., Zhang, X., Ren, S. & Sun, J. Delving deep into rectifiers: surpassing human-level performance on ImageNet classification. In Proc. IEEE International Conference on Computer Vision (ICCV) 1026–1034 (IEEE, 2015).

Yang, Y., Zhang, G., Xu, Z. & Katabi, D. Harnessing structures for value-based planning and reinforcement learning. In Proc. 7th International Conference on Learning Representations (ICLR, 2019).

Lecun, Y., Bottou, L., Bengio, Y. & Haffner, P. Gradient-based learning applied to document recognition. Proc. IEEE 86 , 2278–2324 (1998).

Goodfellow, I., Mirza, M., Xiao, D. & Aaron Courville, Y. B. An empirical investigation of catastrophic forgeting in gradient-based neural networks. In Proc. 2nd International Conference on Learning Representations (ICLR, 2014).

Zenke, F., Poole, B. & Ganguli, S. Continual learning through synaptic intelligence. In Proc. 34th International Conference on Machine Learning 3987–3995 (PMLR, 2017).

Clevert, D., Unterthiner, T. & Hochreiter, S. Fast and accurate deep network learning by exponential linear units (ELUs). In Proc. 4th International Conference on Learning Representations (ICLR, 2016).

Maas, A. L., Hannun, A. Y. & Ng, A. Y. Rectifier nonlinearities improve neural network acoustic models. In Proc. 30th International Conference on Machine Learning (eds Dasgupta, S. & McAllester, D.) (JMLR, 2013).

Nair, V. & Hinton, G. E. Rectified linear units improve restricted Boltzmann machines. In Proc. 27th International Conference on Machine Learning 807–814 (Omnipress, 2010).

Ramachandran, P., Zoph, B. & Le, Q. V. Searching for activation functions. In Proc. 6th International Conference on Learning Representations (eds Murray, I., Ranzato, M. & Vinyals, O.) (ICLR, 2018).

Sutton, R. S. & Whitehead, S. D. Online learning with random representations. In Proc. 10th International Conference on Machine Learning 314–321 (Elsevier, 1993).

Lu, L., Shin, Y., Su, Y. & Karniadakis, G. E. Dying ReLU and initialization: theory and numerical examples. Commun. Computat. Phys . 28 , 1671–1706 (2020).

Shin, Y. & Karniadakis, G. E. Trainability of ReLU networks and data-dependent initialization. J. Mach. Learn. Model. Comput. 1 , 39–74 (2020).

Glorot, X. & Bengio, Y. Understanding the difficulty of training deep feedforward neural networks. In Proc. 13th International Conference on Artificial Intelligence and Statistics 249–256 (PMLR, 2010).

Montavon, G., Orr, G. & Müller, K.-R. Neural Networks: Tricks of the Trade (Springer, 2012).

Rakitianskaia, A. & Engelbrecht, A. Measuring saturation in neural networks. In Proc. 2015 IEEE Symposium Series on Computational Intelligence 1423–1430 (2015).

Boyd, S. P. & Vandenberghe, L. Convex Optimization (Cambridge Univ. Press, 2004).

Roy, O. & Vetterli, M. The effective rank: a measure of effective dimensionality. In Proc. 2007 15th European Signal Processing Conference 606–610 (IEEE, 2007).

Smith, S. L., Dherin, B., Barrett, D. & De, S. On the origin of implicit regularization in stochastic gradient descent. In Proc. 9th International Conference on Learning Representations (ICLR, 2021).

Razin, N. & Cohen, N. Implicit regularization in deep learning may not be explainable by norms. Adv. Neural Inf. Process. Syst. 33 , 21174–21187 (2020).

Goodfellow, I., Bengio, Y. & Courville, A. Deep Learning (MIT Press, 2016).

Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I. & Salakhutdinov, R. Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15 , 1929–1958 (2014).

Bjorck, N., Gomes, C. P., Selman, B. & Weinberger, K. Q. Understanding batch normalization. Adv. Neural Inf. Process. Syst. 31 , 7694–7705 (2018).

Ioffe, S. & Szegedy, C. Batch normalization: accelerating deep network training by reducing internal covariate shift. In Proc. 32nd International Conference on Machine Learning 448–456 (PMLR, 2015).

Chiley, V. et al. Online normalization for training neural networks. Adv. Neural Inf. Process. Syst. 32 , 1–11 (2019).

ADS   Google Scholar  

Ceron, J. S. O. & Castro, P. S. Revisiting rainbow: promoting more insightful and inclusive deep reinforcement learning research. In Proc. 38th International Conference on Machine Learning 1373–1383 (PMLR, 2021).

Brockman, G. et al. OpenAI Gym. Preprint at https://arxiv.org/abs/1606.01540 (2016).

Patterson, A., Neumann, S., White, M. & White, A. Empirical design in reinforcement learning. Preprint at https://arxiv.org/abs/2304.01315 (2023).

Igl, M., Farquhar, G., Luketina, J., Boehmer, W. & Whiteson, S. Transient non-stationarity and generalisation in deep reinforcement learning. In Proc. 9th International Conference on Learning Representations (ICLR, 2021).

Kumar, A., Agarwal, R., Ghosh, D. & Levine, S. Implicit under-parameterization inhibits data-efficient deep reinforcement learning. In Proc. 9th International Conference on Learning Representations (ICLR, 2021).

Nikishin, E. et al. Deep reinforcement learning with plasticity injection. Adv. Neural Inf. Process. Syst. 36 , 1–18 (2023).

D’Oro, P. et al. Sample-efficient reinforcement learning by breaking the replay ratio barrier. In Proc. 11th International Conference on Learning Representations (ICLR, 2023).

Schwarzer, M. et al. Bigger, better, faster: human-level Atari with human-level efficiency. In Proc. 40th International Conference on Machine Learning 30365–30380 (PMLR, 2023).

Lee, H. et al. PLASTIC: improving input and label plasticity for sample efficient reinforcement learning. Adv. Neural Inf. Process. Syst. 36 , 1–26 (2023).

CAS   Google Scholar  

Delfosse, Q., Schramowski, P., Mundt, M., Molina, A. & Kersting, K. Adaptive rational activations to boost deep reinforcement learning. In Proc. 12th International Conference on Learning Representations (ICLR, 2024).

Caruana, R. Multitask learning. Mach. Learn. 28 , 41–75 (1997).

Ring, M. B. in Learning to Learn (eds Thrun, S. & Pratt, L.) 261–292 (Springer, 1998).

Parisi, G. I., Kemker, R., Part, J. L., Kanan, C. & Wermter, S. Continual lifelong learning with neural networks: a review. Neural Netw. 113 , 54–71 (2019).

Kumar, S. et al. Continual learning as computationally constrained reinforcement learning. Preprint at https://arxiv.org/abs/2307.04345 (2023).

Yoon, J., Yang, E., Lee, J. & Hwang, S. J. Lifelong learning with dynamically expandable networks. In Proc. 6th International Conference on Learning Representations (eds Murray, I., Ranzato, M. & Vinyals, O.) (ICLR, 2018).

Aljundi, R. Online continual learning with maximal interfered retrieval. Adv. Neural Inf. Process. Syst. 32 , 1–12 (2019).

Golkar, S., Kagan, M. & Cho, K. in Proc. NeurIPS 2019 Workshop on Real Neurons & Hidden Units: Future Directions at the Intersection of Neuroscience and Artificial Intelligence 146 (NeurIPS, 2019).

Riemer, M. et al. Learning to learn without forgetting by maximizing transfer and minimizing interference. In Proc. 7th International Conference on Learning Representations (ICLR, 2019).

Rajasegaran, J., Hayat, M., Khan, S. H., Khan, F. & Shao, L. Random path selection for continual learning. Adv. Neural Inf. Process. Syst. 32 , 1–11 (2019).

Javed, K. & White, M. Meta-learning representations for continual learning. Adv. Neural Inf. Process. Syst. 32 , 1–11 (2019).

Veniat, T., Denoyer, L. & Ranzato, M. Efficient continual learning with modular networks and task-driven priors. In Proc. 9th International Conference on Learning Representations (ICLR, 2021).

Verwimp, E. et al. Continual learning: applications and the road forward. Trans. Mach. Learn. Res. https://openreview.net/forum?id=axBIMcGZn9 (2024).

Lopez-Paz, D. & Ranzato, M. Gradient episodic memory for continual learning. Adv. Neural Inf. Process. Syst. 30 , 1–10 (2017).

Rusu, A. A. et al. in Proc. 1st Annual Conference on Robot Learning 262–270 (PMLR, 2017).

Chen, J., Nguyen, T., Gorur, D. & Chaudhry, A. Is forgetting less a good inductive bias for forward transfer? In Proc. 11th International Conference on Learning Representations (ICLR, 2023).

Lewandowski, A., Tanaka, H., Schuurmans, D. & Machado, M. C. Directions of curvature as an explanation for loss of plasticity. Preprint at https://arxiv.org/abs/2312.00246 (2024).

Lyle, C. et al. Disentangling the causes of plasticity loss in neural networks. Preprint at https://arxiv.org/abs/2402.18762 (2024).

LeCun, Y., Denker, J. & Solla, S. Optimal brain damage. Adv. Neural Inf. Process. Syst. 2 , 598–605 (1989).

Han, S., Mao, H. & Dally, W. J. Deep compression: compressing deep neural networks with pruning, trained quantization and Huffman coding. In Proc. 4th International Conference on Learning Representations (ICLR, 2016).

Gale, T., Elsen, E. & Hooker, S. The state of sparsity in deep neural networks. Preprint at https://arxiv.org/abs/1902.09574 (2019).

Liu, J., Xu, Z., Shi, R., Cheung, R. C. C. & So, H. K. H. Dynamic sparse training: find efficient sparse network from scratch with trainable masked layers. In Proc. 8th International Conference on Learning Representations (ICLR, 2020).

Elsayed, M. & Mahmood, A. R. Addressing catastrophic forgetting and loss of plasticity in neural networks. In Proc. 12th International Conference on Learning Representations (ICLR, 2024).

Mocanu, D. C. et al. Scalable training of artificial neural networks with adaptive sparse connectivity inspired by network science. Nat. Commun. 9 , 2383 (2018).

Article   ADS   PubMed   Google Scholar  

Bellec, G., Kappel, D., Maass, W. & Legenstein, R. Deep rewiring: training very sparse deep networks. In Proc. 6th International Conference on Learning Representations (eds Murray, I., Ranzato, M. & Vinyals, O.) (ICLR, 2018).

Evci, U., Gale, T., Menick, J., Castro, P. S. & Elsen, E. Rigging the lottery: making all tickets winners. In Proc. 37th International Conference on Machine Learning 2943–2952 (PMLR, 2020).

Chen, T. et al. Chasing sparsity in vision transformers: an end-to-end exploration. Adv. Neural Inf. Process. Syst. 34 , 1–15 (2021).

Sokar, G., Mocanu, E., Mocanu, D. C., Pechenizkiy, M. & Stone, P. Dynamic sparse training for deep reinforcement learning. In Proc. 31st International Joint Conference on Artificial Intelligence (IJCAI-22) (ed. De Raedt, L.) 3437–3443 (IJCAI, 2022).

Graesser, L., Evci, U., Elsen, E. & Castro, P. S. The state of sparse training in deep reinforcement learning. In Proc. 39th International Conference on Machine Learning 7766–7792 (PMLR, 2022).

Zhou, G., Sohn, K. & Lee, H. Online incremental feature learning with denoising autoencoders. In Proc. 15th International Conference on Artificial Intelligence and Statistics 1453–1461 (PMLR, 2012).

Rusu, A. A. et al. Progressive neural networks. Preprint at https://arxiv.org/abs/1606.04671 (2022).

Sutskever, I., Martens, J., Dahl, G. & Hinton, G. On the importance of initialization and momentum in deep learning. In Proc. 30th International Conference on Machine Learning (eds Dasgupta, S. & McAllester, D.) 1139–1147 (JMLR, 2013).

Frankle, J. & Carbin, M. The lottery ticket hypothesis: finding sparse, trainable neural networks. In Proc. 7th International Conference on Learning Representations (ICLR, 2019).

Finn, C., Abbeel, P. & Levine, S. Model-agnostic meta-learning for fast adaptation of deep networks. In Proc. 34th International Conference on Machine Learning 1126–1135 (PMLR, 2017).

Wang, Y.-X., Ramanan, D. & Hebert, M. Growing a brain: fine-tuning by increasing model capacity. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2471–2480 (IEEE, 2017).

Nagabandi, A. et al. Learning to adapt in dynamic, real-world environments through meta-reinforcement learning. In Proc. 7th International Conference on Learning Representations (ICLR, 2019).

Holmstrom, L. & Koistinen, P. et al. Using additive noise in back-propagation training. IEEE Trans. Neural Netw. 3 , 24–38 (1992).

Graves, A., Mohamed, A.-R. & Hinton, G. Speech recognition with deep recurrent neural networks. In Proc. 2013 IEEE International Conference on Acoustics, Speech and Signal Processing 6645–6649 (IEEE, 2013).

Neelakantan, A. et al. Adding gradient noise improves learning for very deep networks. Preprint at https://arxiv.org/abs/1511.06807 (2015).

Greff, K., Srivastava, R. K., Koutník, J., Steunebrink, B. R. & Schmidhuber, J. LSTM: a search space odyssey. IEEE Trans. Neural Netw. Learn. Syst. 28 , 2222–2232 (2017).

Article   MathSciNet   PubMed   Google Scholar  

Download references

Acknowledgements

We thank M. White for her feedback on an earlier version of this work; P. Nagarajan, E. Graves, G. Mihucz, A. Hakhverdyan, K. Roice, T. Ferguson, L. Watson, H. Sinha, P. Bhangale and M. Przystupa for their feedback on writing; and M. C. Machado for encouraging us to make this work accessible to a general scientific audience. We gratefully acknowledge the Digital Research Alliance of Canada for providing the computational resources to carry out the experiments in this paper. We also acknowledge funding from the Canada CIFAR AI Chairs program, DeepMind, the Alberta Machine Intelligence Institute (Amii), CIFAR and the Natural Sciences and Engineering Research Council of Canada (NSERC). This work was made possible by the stimulating and supportive research environment created by the members of the Reinforcement Learning and Artificial Intelligence (RLAI) laboratory, particularly within the agent-state research meetings.

Author information

Authors and affiliations.

Department of Computing Science, University of Alberta, Edmonton, Alberta, Canada

Shibhansh Dohare, J. Fernando Hernandez-Garcia, Qingfeng Lan, Parash Rahman, A. Rupam Mahmood & Richard S. Sutton

Canada CIFAR AI Chair, Alberta Machine Intelligence Institute (Amii), Edmonton, Alberta, Canada

A. Rupam Mahmood & Richard S. Sutton

You can also search for this author in PubMed   Google Scholar

Contributions

S.D., J.F.H.-G., Q.L. and A.R.M. wrote the software. S.D., J.F.H.-G. and P.R. prepared the datasets. S.D. and J.F.H.-G. designed the experiments. S.D., J.F.H.-G., Q.L., R.S.S. and A.R.M. analysed and interpreted the results. S.D., A.R.M. and R.S.S. developed the continual backpropagation algorithm. S.D., J.F.H.-G., Q.L., R.S.S. and A.R.M. prepared the manuscript.

Corresponding author

Correspondence to Shibhansh Dohare .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Peer review

Peer review information.

Nature thanks Pablo Castro, Razvan Pascanu and Gido van de Ven for their contribution to the peer review of this work. Peer reviewer reports are available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data figures and tables

Extended data fig. 1 further results on class-incremental cifar-100..

a , Test accuracy in class-incremental CIFAR-100. As more classes are added, the classification becomes harder and algorithms naturally show decreasing accuracy with more classes. Each line corresponds to the average of 15 runs. b , Test accuracy of continual backpropagation for different values of the replacement rate parameter with contribution utility and 1,000 maturity threshold. The line corresponding to 10 −4 is an average of five runs, whereas the other two lines are an average of 15 runs. The solid lines represent the mean and the shaded regions correspond to ±1 standard error.

Extended Data Fig. 2 Loss of plasticity in the Slowly-Changing Regression problem.

a , The target function and the input in the Slowly-Changing Regression problem. The input has m  + 1 bits. One of the flipping bits is chosen after every T time steps and its value is flipped. The next m  −  f bits are i.i.d. at every time step and the last bit is always one. The target function is represented by a neural network with a single hidden layer of LTUs. Each weight in the target network is −1 or 1. b , Loss of plasticity is robust across different activations. These results are averaged over 100 runs; the solid lines represent the mean and the shaded regions correspond to ±1 standard error.

Extended Data Fig. 3 Loss of plasticity in Online Permuted MNIST.

a , Left, an MNIST image with the label ‘7’; right, a corresponding permuted image. b , Loss of plasticity in Online Permuted MNIST is robust over step sizes, network sizes and rates of change. c , Evolution of various qualities of a deep network trained by means of backpropagation with different step sizes. Left, over time, the percentage of dead units in the network increases. Centre, the average magnitude of the weights increases over time. Right, the effective rank of the representation of the networks trained with backpropagation decreases over time. The results in these six plots are the average over 30 runs. The solid lines represent the mean and the shaded regions correspond to ±1 standard error. For some lines, the shaded region is thinner than the line width, as standard error is small.

Extended Data Fig. 4 Existing deep-learning methods on Online Permuted MNIST.

a , Left, online classification accuracy of various algorithms on Online Permuted MNIST. Shrink and Perturb has almost no drop in online classification accuracy over time. Continual backpropagation did not show any loss of plasticity and had the best level of performance. Centre left, over time, the percentage of dead units increases in all methods except for continual backpropagation; it has almost zero dead units throughout learning. Centre right, the average magnitude of the weights increases over time for all methods except for L2 regularization, Shrink and Perturb and continual backpropagation. These are also the three best-performing methods, which suggests that small weights are important for fast learning. Right, the effective rank of the representation of all methods drops over time. However, continual backpropagation maintains a higher effective rank than both backpropagation and Shrink and Perturb. Among all the algorithms, only continual backpropagation maintains a high effective rank, low weight magnitude and low percentage of dead units. The results correspond to the average over 30 independent runs. The shaded regions correspond to ±1 standard error. b , Performance of various algorithms on Online Permuted MNIST for various hyperparameter combinations. For each method, we show three different hyperparameter settings. The parameter settings that were used in the left panel in a are marked with a solid square next to their label. The results correspond to the average of over 30 runs for settings marked with a solid square and 10 runs for the rest. The solid lines represent the mean and the shaded regions correspond to ±1 standard error.

Extended Data Fig. 5 Further results in stationary reinforcement-learning problems.

a , Similar to Fig. 4 , the performance of standard PPO drops over time. However, unlike in Fig. 4 , the performance of PPO with L2 regularization gets worse over time in Hopper-v3. On the other hand, PPO with continual backpropagation and L2 regularization can keep improving with time. b , Comparison of continual backpropagation and ReDo on Ant-v3. The performance of PPO with ReDo and L2 regularization worsens over time, whereas PPO with continual backpropagation and L2 regularization keeps improving over time. c , PPO with standard Adam leads to large updates in the policy network compared with proper Adam ( β 1  =  β 1  = 0.99), which explains why PPO with proper Adam performs much better than standard PPO. d , Comparison of two forms of utility in continual backpropagation, when using a running estimate of instantaneous utility and when using just the instantaneous utility. Both variations have similar performance. All these results are averaged over 30 runs; the solid lines represent the mean and the shaded regions correspond to 95% bootstrapped confidence interval.

Supplementary information

Peer review file, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Dohare, S., Hernandez-Garcia, J.F., Lan, Q. et al. Loss of plasticity in deep continual learning. Nature 632 , 768–774 (2024). https://doi.org/10.1038/s41586-024-07711-7

Download citation

Received : 11 August 2023

Accepted : 12 June 2024

Published : 21 August 2024

Issue Date : 22 August 2024

DOI : https://doi.org/10.1038/s41586-024-07711-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

• Engineering Mathematics
• Discrete Mathematics
• Operating System
• Computer Networks
• Digital Logic and Design
• C Programming
• Data Structures
• Theory of Computation
• Compiler Design
• Computer Org and Architecture

Types of Network Topology

Network topology refers to the arrangement of different elements like nodes, links, or devices in a computer network. It defines how these components are connected and interact with each other. Understanding various types of network topologies helps in designing efficient and robust networks. Common types include bus, star, ring, mesh, and tree topologies, each with its own advantages and disadvantages. In this article, we are going to discuss different types of network topology their advantages and disadvantages in detail.

The arrangement of a network that comprises nodes and connecting lines via sender and receiver is referred to as Network Topology . The various network topologies are:

Point to Point Topology

Mesh Topology

Star Topology

Bus Topology

Ring Topology

Tree Topology

Hybrid Topology

Point-to-point topology is a type of topology that works on the functionality of the sender and receiver. It is the simplest communication between two nodes, in which one is the sender and the other one is the receiver. Point-to-Point provides high bandwidth.

In a mesh topology, every device is connected to another device via a particular channel. In Mesh Topology, the protocols used are AHCP (Ad Hoc Configuration Protocols), DHCP (Dynamic Host Configuration Protocol), etc.

Figure 1 : Every device is connected to another via dedicated channels. These channels are known as links. 

• Suppose, the N number of devices are connected with each other in a mesh topology, the total number of ports that are required by each device is N-1. In Figure 1, there are 5 devices connected to each other, hence the total number of ports required by each device is 4. The total number of ports required = N * (N-1).
• Suppose, N number of devices are connected with each other in a mesh topology, then the total number of dedicated links required to connect them is N C 2 i.e. N(N-1)/2. In Figure 1, there are 5 devices connected to each other, hence the total number of links required is 5*4/2 = 10.

Advantages of Mesh Topology

• Communication is very fast between the nodes.
• Mesh Topology is robust.
• The fault is diagnosed easily. Data is reliable because data is transferred among the devices through dedicated channels or links.
• Provides security and privacy.

Disadvantages of Mesh Topology

• Installation and configuration are difficult.
• The cost of cables is high as bulk wiring is required, hence suitable for less number of devices.
• The cost of maintenance is high.

A common example of mesh topology is the internet backbone, where various internet service providers are connected to each other via dedicated channels. This topology is also used in military communication systems and aircraft navigation systems.

For more, refer to the Advantages and Disadvantages of Mesh Topology .

In Star Topology, all the devices are connected to a single hub through a cable. This hub is the central node and all other nodes are connected to the central node. The hub can be passive in nature i.e., not an intelligent hub such as broadcasting devices, at the same time the hub can be intelligent known as an active hub. Active hubs have repeaters in them. Coaxial cables or RJ-45 cables are used to connect the computers. In Star Topology, many popular Ethernet LAN protocols are used as CD(Collision Detection), CSMA (Carrier Sense Multiple Access), etc.

Figure 2 : A star topology having four systems connected to a single point of connection i.e. hub. 

Advantages of Star Topology

• If N devices are connected to each other in a star topology, then the number of cables required to connect them is N. So, it is easy to set up.
• Each device requires only 1 port i.e. to connect to the hub, therefore the total number of ports required is N.
• It is Robust. If one link fails only that link will affect and not other than that.
• Easy to fault identification and fault isolation.
• Star topology is cost-effective as it uses inexpensive coaxial cable.

Disadvantages of Star Topology

• If the concentrator (hub) on which the whole topology relies fails, the whole system will crash down.
• The cost of installation is high.
• Performance is based on the single concentrator i.e. hub.

A common example of star topology is a local area network (LAN) in an office where all computers are connected to a central hub. This topology is also used in wireless networks where all devices are connected to a wireless access point.

For more, refer to the Advantages and Disadvantages of Star Topology.

Bus Topology is a network type in which every computer and network device is connected to a single cable. It is bi-directional. It is a multi-point connection and a non-robust topology because if the backbone fails the topology crashes. In Bus Topology, various MAC (Media Access Control) protocols are followed by LAN ethernet connections like TDMA , Pure Aloha , CDMA, Slotted Aloha , etc.

Figure 3 : A bus topology with shared backbone cable. The nodes are connected to the channel via drop lines. 

Advantages of Bus Topology

• If N devices are connected to each other in a bus topology, then the number of cables required to connect them is 1, known as backbone cable, and N drop lines are required.
• Coaxial or twisted pair cables are mainly used in bus-based networks that support up to 10 Mbps.
• The cost of the cable is less compared to other topologies, but it is used to build small networks.
• Bus topology is familiar technology as installation and troubleshooting techniques are well known.
• CSMA is the most common method for this type of topology.

 Disadvantages of  Bus Topology

• A bus topology is quite simpler, but still, it requires a lot of cabling.
• If the common cable fails, then the whole system will crash down.
• If the network traffic is heavy, it increases collisions in the network. To avoid this, various protocols are used in the MAC layer known as Pure Aloha, Slotted Aloha, CSMA/CD, etc.
• Adding new devices to the network would slow down networks.
• Security is very low.

A common example of bus topology is the Ethernet LAN, where all devices are connected to a single coaxial cable or twisted pair cable. This topology is also used in cable television networks. For more, refer to the Advantages and Disadvantages of Bus Topology .

In a Ring Topology, it forms a ring connecting devices with exactly two neighboring devices. A number of repeaters are used for Ring topology with a large number of nodes, because if someone wants to send some data to the last node in the ring topology with 100 nodes, then the data will have to pass through 99 nodes to reach the 100th node. Hence to prevent data loss repeaters are used in the network.

The data flows in one direction, i.e. it is unidirectional, but it can be made bidirectional by having 2 connections between each Network Node, it is called Dual Ring Topology. In-Ring Topology, the Token Ring Passing protocol is used by the workstations to transmit the data.

Figure 4 : A ring topology comprises 4 stations connected with each forming a ring. 

The most common access method of ring topology is token passing.

• Token passing: It is a network access method in which a token is passed from one node to another node.
• Token: It is a frame that circulates around the network.

Operations of Ring Topology

• One station is known as a monitor station which takes all the responsibility for performing the operations.
• To transmit the data, the station has to hold the token. After the transmission is done, the token is to be released for other stations to use.
• When no station is transmitting the data, then the token will circulate in the ring.
• There are two types of token release techniques: Early token release releases the token just after transmitting the data and Delayed token release releases the token after the acknowledgment is received from the receiver.

Advantages of Ring Topology

• The data transmission is high-speed.
• The possibility of collision is minimum in this type of topology.
• Cheap to install and expand.
• It is less costly than a star topology.

Disadvantages of Ring Topology

• The failure of a single node in the network can cause the entire network to fail.
• Troubleshooting is difficult in this topology.
• The addition of stations in between or the removal of stations can disturb the whole topology.
• Less secure. 

For more, refer to the Advantages and Disadvantages of Ring Topology .

This topology is the variation of the Star topology. This topology has a hierarchical flow of data. In Tree Topology, protocols like DHCP and SAC (Standard Automatic Configuration ) are used.

Figure 5 : In this, the various secondary hubs are connected to the central hub which contains the repeater. This data flow from top to bottom i.e. from the central hub to the secondary and then to the devices or from bottom to top i.e. devices to the secondary hub and then to the central hub. It is a multi-point connection and a non-robust topology because if the backbone fails the topology crashes.

Advantages of Tree Topology

• It allows more devices to be attached to a single central hub thus it decreases the distance that is traveled by the signal to come to the devices.
• It allows the network to get isolated and also prioritize from different computers.
• We can add new devices to the existing network.
• Error detection and error correction are very easy in a tree topology.

Disadvantages of Tree Topology

• If the central hub gets fails the entire system fails.
• The cost is high because of the cabling.
• If new devices are added, it becomes difficult to reconfigure.

A common example of a tree topology is the hierarchy in a large organization. At the top of the tree is the CEO, who is connected to the different departments or divisions (child nodes) of the company. Each department has its own hierarchy, with managers overseeing different teams (grandchild nodes). The team members (leaf nodes) are at the bottom of the hierarchy, connected to their respective managers and departments.

For more, refer to the Advantages and Disadvantages of Tree Topology .

This topological technology is the combination of all the various types of topologies we have studied above. Hybrid Topology is used when the nodes are free to take any form. It means these can be individuals such as Ring or Star topology or can be a combination of various types of topologies seen above. Each individual topology uses the protocol that has been discussed earlier.

The above figure shows the structure of the Hybrid topology. As seen it contains a combination of all different types of networks.

Advantages of Hybrid Topology

• This topology is very flexible .
• The size of the network can be easily expanded by adding new devices.

Disadvantages of Hybrid Topology

• It is challenging to design the architecture of the Hybrid Network.
• Hubs used in this topology are very expensive.
• The infrastructure cost is very high as a hybrid network requires a lot of cabling and network devices .

A common example of a hybrid topology is a university campus network. The network may have a backbone of a star topology, with each building connected to the backbone through a switch or router. Within each building, there may be a bus or ring topology connecting the different rooms and offices. The wireless access points also create a mesh topology for wireless devices. This hybrid topology allows for efficient communication between different buildings while providing flexibility and redundancy within each building.

For more, refer to the Advantages and Disadvantages of Hybrid Topology .

In conclusion, network topologies play a crucial role in determining the efficiency and reliability of a computer network. Each topology, whether it’s bus, star, ring, mesh, or tree, offers unique benefits and potential drawbacks. By understanding these different arrangements, network designers can choose the most appropriate topology to meet the specific needs of their systems, ensuring optimal performance and connectivity.

Frequently Asked Questions on Network Topology – FAQs

What is the main benefit of tree topology.

Tree topology combines characteristics of star and bus topologies. It supports future expandability of the network and provides efficient data management

Which topology is best for large networks?

For large networks, mesh and tree topologies are often preferred. Mesh topology offers high reliability and redundancy, while tree topology supports scalability and efficient data organization.

Can different topologies be combined in a single network?

Yes, different topologies can be combined in a hybrid topology to take advantage of the strengths of each type, improving overall network performance and reliability.

How do I choose the right network topology for my needs?

Choosing the right network topology depends on factors such as the size of your network, budget, desired performance, and the need for reliability and scalability. Assess your specific requirements to make an informed decision.

Please Login to comment...

Similar reads.

• Data Link Layer

Improve your Coding Skills with Practice

What kind of Experience do you want to share?

Utilization of Artificial Intelligence to Improve Door-In Door-Out Times for Mechanical Thrombectomy-Eligible Patients at a Hub-and-Spoke Community-Based Comprehensive Stroke Center: A Single Case Study Presentation AI Improving DIDO Times

Article sidebar, main article content.

Background:

Delays in the transfer of patients with hyperacute stroke may lead to treatment ineligibility due to the degree of cerebral hypoxic injury suffered. Cloud-based artificial intelligence applications may improve transfer times and expand access to advanced therapies. One case between a community-based comprehensive stroke center (CSC) and a primary stroke center (PSC) using a third-party telemedicine service and shared cloud-based artificial intelligence application may provide insight in opportunities to improve stroke systems of care. 

  Case Presentation:

A 62-year-old female with a past medical history of hypertension, current everyday tobacco smoker, and marijuana user presented to an outlying emergency department (ED) with dense left-sided hemiplegia affecting the arm and leg, right-sided gaze preference, and severe dysarthria. Her last known well (LKW) time was 0900 hours. CNS imaging revealed a right middle cerebral artery occlusion, visible to members of the CSC stroke team through the use of a cloud-based artificial intelligence cell phone application. The patient was treated with intravenous thrombolytics at the PSC, and she was transferred to the CSC, where she underwent a diagnostic cerebral arteriogram with carotid artery stenting. Later, Magnetic Resonance Imaging (MRI) of the brain revealed a 3.5 cm x 2.5 cm hemorrhagic lesion in the right frontal lobe and diffusion restriction in the right frontal and right posterior temporal lobes. The patient’s hospital stay was three days and, at the time of discharged, her modified Rankin score and NIHSS were zero. She was discharged on dual antiplatelet therapy, statin therapy, and nicotine replacement. 

Utilization of Artificial Intelligence:

            Transfer delays are complicated by organizing care at PSC and CSC and can be lengthy when communication across different facilities and subspecialties. Implementing cloud-based AI image sharing in stroke systems of care has reduced DIDO times by providing rapid imaging interpretation, streamlining communication, and enhancing coordination between PSCs and CSCs. 

Conclusions:

Our case presentation showed how a hub-and-spoke model combined with cloud-based AI utilization can improve DIDO times and enhance stroke systems of care.

Article Details

This work is licensed under a Creative Commons Attribution 4.0 International License .

Quantifying and Analyzing the Uncertainty in Fault Interpretation Using Entropy

• Published: 23 August 2024

Cite this article

• Zhicheng Lei   ORCID: orcid.org/0000-0002-3407-9859 1 , 2  

Fault interpretation in geology inherently involves uncertainty, which has driven the need to develop methods to quantify and analyze this uncertainty. This paper introduces a novel framework for this task by integrating graph theory, entropy, and random walk. The proposed approach employs graph theory to mathematically represent a fault network in both map-view and profile sections. By integrating the theory of two-dimensional random walk, the stochastic nature of the fault growth process can be effectively characterized, enabling the development of tailored probability formulations for the fault network through weighted graph theory. In addition, entropy models tailored to the fault network are formulated, providing a solid foundation for uncertainty quantification and analysis. Furthermore, the proposed method employs the principle of increase of entropy to quantitatively assess the uncertainty involved in comparing different fault networks. A case study is presented to demonstrate the practical application in addressing the challenges associated with quantifying, communicating, and analyzing the uncertainty in fault interpretation. The findings obtained in this study suggest that (1) entropy serves as a reliable metric for measuring and communicating the uncertainty in fault interpretation; (2) entropy can be used to estimate the potential numbers of evolutionary paths available for a fault network; and (3) the growth process of a fault network adheres to the principle of increase of entropy, enabling us to utilize entropy to measure the complexity of the fault network and subsequently compare the differences between various fault networks. The results obtained highlight the potential of this approach not only for understanding the geological meaning of uncertainty in fault interpretation but also for enhancing decision-making in related fields.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Antonucci A, de Campos CP, Zaffalon M (2014) Probabilistic graphical models. In: Augustin T, Coolen FPA, de Cooman G, Troffaes MCM (eds) Academic press library in signal processing: volume 1 signal processing theory and machine learning, Elsevier. https://doi.org/10.1002/9781118763117.ch9

Baugh J, Moussa O, Ryan CA, Nayak A, Laflamme R (2005) Experimental implementation of heat-bath algorithmic cooling using solid-state nuclear magnetic resonance. Nature 438:470–473

Article   CAS   Google Scholar  

Bekenstein JD (2007) Information in the holographic universe. Sci Am Rep 289:67–73

Google Scholar  

Ben-Naim A (2017) Entropy, Shannon’s measure of information and Boltzmann’s H-theorem. Entropy 19:48. https://doi.org/10.3390/e19020048

Article   Google Scholar  

Bertsekas DP, Tsitsiklis JN (2008) Introduction to probability, 2nd edn. Massachusetts Institute of Technology, Cambridge

Bérut A, Arakelyan A, Petrosyan A, Ciliberto S, Dillenschneider R, Lutz E (2012) Experimental verification of Landauer’s principle linking information and thermodynamics. Nature 483(7388):187–189

Bizhani M, Kuru E (2022) Towards drilling rate of penetration prediction: Bayesian neural networks for uncertainty quantification. J Petrol Sci Eng 219:111068

Bond CE (2015) Uncertainty in structural interpretation: lessons to be learnt. J Struct Geol 74:185–200. https://doi.org/10.1016/j.jsg.2015.03.003

Butler RWH, Bond CE, Cooper MA, Watkins H (2018) Interpreting structural geometry in fold-thrust belts: Why style matters? J Struct Geol 114:251–273

Cardenas I, Aven T, Flage R (2023) Addressing challenges in uncertainty quantification: the case of geohazard assessments. Geosci Model Dev 16(6):1601–1615. https://doi.org/10.5194/gmd-16-1601-2023

Cover TM, Thomas JA (2006) Elements of information theory, 2nd edn. Wiley, New York, pp 81–84

Delogkos E, Childs C, Manzocchi T, Wash JJ, Pavlides S (2017) The role of bed-parallel slip in the development of complex normal fault zones. J Struct Geol 97:199–211. https://doi.org/10.1016/j.jsg.2017.02.014

Delogkos E, Manzocchi T, Childs C, Camanni G, Roche V (2020) The 3D structural of a normal fault from multiple outcrop observations. J Struct Geol 136:104009. https://doi.org/10.1016/j.jsg.2020.104009

Dooley TP, Schreurs G (2012) Analogue modelling of interpolate strike-slip tectonics: a review and new experimental results. Tectonophysics 574–575(29):1–71

Fedorik J, Zwaan F, Scjreirs G, Toscani G, Bonini L, Seno S (2019) The interaction between strike-slip dominated fault zones and thrust belt structures: insights from 4D analogue models. J Struct Geol 122:89–105

Feynman RP, Leighton RB, Sands M (1964) The Feynman lectures on physics, vol. 1, chapter 46: Ratchet and Pawl. Addison-Wesley, London

Flottmann T, Hand M, Close D, Edgoose C, Scrimgeour I (2004) Thrust tectonic styles of the intracratonic Alice Springs and Petermann Orogenies, Central Australia. In: McClay KR (ed) Thrust tectonics and hydrocarbon systems. AAPG Memoir 82, American Association of Petroleum Geologists, pp 538–557

Freeman B, Boult PJ, Yielding G, Menpes S (2010) Using empirical geological rules to reduce structural uncertainty in seismic interpretation of faults. J Struct Geol 32:1668–1676

Gabrieli T, Hatoz YH, Sagy A (2024) Kinematics and geometry of normal faults at the margins of pull-apart basins: an example from the Dead Sea Basin. Tectonophysics 870:230156

Gupta A, Govindaraju RS (2019) Propagation of structural uncertainty in watershed hydrologic models. J Hydrol 575:66–81

Gupta A, Govindaraju RS (2023) Uncertainty quantification in watershed hydrology: Which method to use? J Hydrol 616:128749

Gupta A, Govindaraju RS, Morbidelli R, Corradini C (2022) The role of prior probabilities on parameter estimation in hydrological models. Water Resour Res 58(5):e2021WR031291

Hoteit H, He X, Yan B, Vahrenkamp V (2023) Uncertainty quantification and optimization method applied to time-continuous geothermal energy extraction. Geothermics 110:102675

Huang J, Deng H, Chen J, Li N, Wang J, Liu Z, Mao X (2023) Assessing geometrical uncertainties in geological interface models using Markov chain Monte Carlo sampling via abstract graph. Tectonophysics 864:230032

Hughes AN, Shaw JH (2014) Fault displacement-distance relationships as indicators of contractional fault-related folding style. AAPG Bull 98(2):227–251

Jackson CAL, Bell RE, Rotevatn A, Tvedt ABM (2017) Techniques to determine the kinematics of synsedimentary normal faults and implications for fault growth models. Geol Soc Lond Spec Publ 439(1):187–217. https://doi.org/10.1144/SP439.2

Jagger LJ, McClay KR (2018) Analogue modelling of inverted domino-style basement fault systems. Basin Res 30:363–381. https://doi.org/10.1111/bre.12224

Jordan MI (2003) An introduction to probabilistic graphical models. Research Gate. https://www.researchgate.net/publication/243774881_An_Introduction_to_Probabilistic_Graphical_Models

Klafter J, Sokolov IM (2011) First steps in random walks: from tools to applications. Oxford University Press, Oxford

Book   Google Scholar  

Krajnovich A, Zhou W, Gutierrez M (2020) Uncertainty assessment for 3D geological modeling of fault zones based on geologic inputs and prior knowledge. Solid Earth 11:1457–1474

Landauer R (1961) Irreversibility and heat generation in the computing process. IBM J Res Dev 5(3):183–191. https://doi.org/10.1147/rd.53.0183

Lindsay MD, Ailleres L, Jessell MW, de Kemp EA, Betts PG (2012) Locating and quantifying geological uncertainty in three-dimensional models: analysis of the Gippsland Basin, southeastern Australia. Tectonophysics 546–547:10–27

McManus S, Rahman A, Coombes J, Horta A (2021) Uncertainty assessment of spatial domain models in early stage mining projects—a review. Ore Geol Rev 133:104098

Mitra S, Paul D (2011) Structural geometry and evolution of restraining bends: Insights from laser-scanned experimental models. AAPG Bull 95(7):1147–1180. https://doi.org/10.1306/09271010060

Nixon CW, Sanderson DJ, Dee SJ, Bull JM, Humphreys RJ, Swanson MH (2014) Fault interactions and reactivation within a normal-fault network at Milne Point, Alaska. AAPG Bull 98(10):2081–2107

Nyambuya GG (2019) A simple proof of the second law of thermodynamics. Prog Phys 15(3):171–177

CAS   Google Scholar  

Pakyuz-Charrier E, Jessel M, Giraud J, Lindsay M, Ogarko V (2019) Topological analysis in Monte Carlo simulation for uncertainty propagation. Solid Earth 10:1663–1684

Pérez-Díaz L, Alcalde J, Bond CE (2020) Introduction: handling uncertainty in the geosciences: identification, mitigation and communication. Solid Earth 11:889–897

Pernkopf F, Peharz R, Tschiatschek S (2014) Introduction to probabilistic graphical models. In: Diniz PSR, Suykens JAK, Chellappa R, Theodoridis S (eds) Academic Press Library in signal processing, vol 1. Elsevier, Amsterdam, pp 989–1064. https://doi.org/10.1016/B978-0-12-396502-8.00018-8

Chapter   Google Scholar  

Pizzi M, Lonergan L, Whittaker AC, Mayall M (2020) Growth of a thrust fault array in space and time: an example from the deep-water Niger delta. J Struct Geol 137:104088

Poblet J, Lisle RJ (2011) Kinematic evolution and structural styles of fold-and-thrust belts. In: Poblet J, Lisle RJ (eds) Kinematic evolution and structural styles of fold-and-thrust belts. Geological Society, London, Special Publications, vol 349, pp 1–24

Qu Y, Perrin M, Torabi A, Abel M, Giese M (2023) GeoFault: a well-founded fault ontology for interoperability in geological modeling. Comput Geosci 182:105478

Roche V, Homberg C, Rocher M (2013) Fault nucleation, restriction, and aspect ratio in layered sections: quantification of the strength and stiffness roles using numerical modeling. JGR Solid Earth 118:4446–4460

Rosen KH (2012) Discrete mathematics and its applications, 8th edn. McGraw-Hill, New York, pp 673–675

Rotevatn A, Jackson CAL, Tvedt ABM, Bell RC, Blækkan I (2019) How do normal faults grow? J Struct Geol 125:174–184

Schaaf A, Bond CE (2019) Quantification of uncertainty in 3-D seismic interpretation: implications for deterministic and stochastic geomodelling and machine learning. Solid Earth 10:1049–1061

Sen D, Chen H, Datta-Gupta A, Kwon J, Mishra S (2021) Machine learning based rate optimization under geologic uncertainty. J Petrol Sci Eng 207:109116

Sgambato C, Walker JPF, Roberts GP (2020) Uncertainty in strain-rate from field measurements of the geometry, rates and kinematics of active normal faults: implications for seismic hazard assessment. J Struct Geol 131:103934

Shannon CE (1948) A mathematical theory of communication. Bell Syst Tech J 27(3):379–423

Sharma PS, Yadav D, Garg P (2020) A systematic review on page ranking algorithms. Int J Inf Technol 12(2):329–337

Sheldon MS (2014) Introduction to probability models, 11th edn. Academic Press, London

Tartakovsky DM, Nowak W, Bolster D (2012) Special issue on uncertainty quantification and risk assessment. Adv Water Resour 36:1–164

Tavani S, Storti F, Salvini F (2006) Double-edge fault-propagation folding: geometry and kinematics. J Struct Geol 28:19–35

Uddin MG, Nash S, Rahman A, Olbert AI (2023) A novel approach for estimating and predicting uncertainty in water quality index model using machine learning approaches. Water Res 229:119422

Venâncio MB, da Silva FCA (2023) Structures evolution along strike-slip fault zones: the role of rheology revealed by PIV analysis of analog modeling. Tectonophysics 851:229764

Viola G, Odonne F, Mancktelow NS (2004) Analogue modelling of reverse fault reactivation in strike-slip and transpressive regimes: application to the Giudicarie fault system, Italian Eastern Alps. J Struct Geol 36:401–418. https://doi.org/10.1016/j.jsg.2003.08.014

Weaver W (1949) Recent contributions to the mathematical theory of communication. In: Shannon CE, Weaver W (eds) The mathematical theory of communication. University of Illionis Press, Urbana and Chicago

Wei Z, He H, Sun W, Zhuang Q, Liang Z (2020) Investigating thrust-fault growth and segment linkage using displacement distribution analysis in the active Duzhanzi thrust fault zone, Northern TianShan of China. J Struct Geol 133:103990

Wellmann JF, Regenauer-Lieb K (2012) Uncertainties have a meaning: information entropy as a quality measure for 3-D geological models. Tectonophysics 526–529:207–216

Witter JB, Trainor-Guitton WJ, Siler DL (2019) Uncertainty and risk evaluation during the exploration stage of geothermal development: a review. Geothermics 79:233–242

Yao Y, Zeng L, Mao Z, Han J, Cao D, Lin B (2023) Differential deformation of a strike-slip fault in the Paleozoic carbonate reservoirs of the Tarim Basin, China. J Struct Geol 173:104908

Yeh C, Dong J, Khoshnevisanm S, Juang CH, Huang W, Lu Y (2021) The role of the geological uncertainty in a geotechnical design—a retrospective view of Freeway No.3 landslide in Northern Taiwan. Eng Geol 291:106–233

Zhang X, Zhang G, Jiang Y, Cui M, He X (2023) The control of curved faults on structural trap formation: Tiantai slope belt, Xihu Sag, East China Sea. Mar Pet Geol 156:106464

Download references

Acknowledgements

I am thankful to the Department of Information School, East China University of Technology, for providing a free research environment. My sincerest thanks also go to Dr. Eric A. de Kemp and an anonymous reviewer for their insightful observations and constructive suggestions that have greatly improved the quality and clarity of this manuscript. This research was funded by Doctors Foundation of East China University of Technology (No. DHBK2019224).

Author information

Authors and affiliations.

School of Information Engineering, East China University of Technology, Nanchang, 330013, China

Zhicheng Lei

Jiangxi Engineering Technology Research Center of Nuclear Geoscience Data Science and System, East China University of Technology, Nanchang, 330013, China

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Zhicheng Lei .

Ethics declarations

Conflict of interest.

The author declares that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Lei, Z. Quantifying and Analyzing the Uncertainty in Fault Interpretation Using Entropy. Math Geosci (2024). https://doi.org/10.1007/s11004-024-10156-3

Download citation

Received : 29 September 2023

Accepted : 31 July 2024

Published : 23 August 2024

DOI : https://doi.org/10.1007/s11004-024-10156-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Uncertainty quantification
• Fault networks
• Uncertainty analysis
• Random walk
• The principle of increase of entropy
• Find a journal
• Publish with us
• Track your research

Different treatment durations of loperamide in preventing pyrotinib-induced diarrhea: A randomized, parallel-group sub-study of the phase II PHAEDRA trial

• Find this author on Google Scholar
• Find this author on PubMed
• Search for this author on this site
• For correspondence: [email protected]
• ORCID record for Qiang Sun
• Info/History
• Preview PDF

Background: Pyrotinib, a pan-HER tyrosine kinase inhibitor, demonstrates efficacy in the treatment of HER2-positive breast cancer. However, the frequent occurrence of treatment-emergent diarrhea necessitating discontinuation, impacts patient outcomes. Methods: In this multicenter, open-label, phase II PHAEDRA study enrolling early stage HER2-positive patients for postoperative treatment with nab-paclitaxel and pyrotinib, 120 patients were included for a sub-study and randomly divided into two groups to receive 21 days and 42 days of loperamide for primary prophylaxis of diarrhea, followed by as-needed usage. The primary outcome was the incidence of grade ≥3 diarrhea. Results: Fifty-eight patients in the 21-day group and 59 patients in the 42-day group received at least one dose of pyrotinib. With a median follow-up of 12.1 months, all patients experienced diarrhea of any grade, with grade ≥3 events in 39.7% of the 21-day group and 42.4% of the 42-day group (relative risk: 0.94; 95% confidence interval: 0.61-1.45). The most common treatment-emergent adverse events, other than diarrhea, were hypoesthesia, vomiting, nausea, and rash, mostly grade 1-2, except for one case of grade ≥3 decreased neutrophil count in each group. Conclusion: No significant differences were observed between 21-day and 42-day loperamide durations in preventing grade ≥3 diarrhea. Considering the economic cost and patient compliance, 21-day loperamide prophylaxis might represent a more pragmatic and appropriate approach for clinical application.

Competing Interest Statement

The authors have declared no competing interest.

Clinical Trial

NCT04659499

Funding Statement

The study drugs pyrotinib and nab-paclitaxel were provided by Jiangsu Hengrui Pharmaceuticals Co., Ltd. This study was funded by the National High-Level Hospital Clinical Research Funding (No. 2022-PUMCH-B-038 and No. 2022-PUMCH-C-066) and TSC China Allicace (#LAM001-202205).

Author Declarations

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

The study was approved by the institutional ethics committee at the Peking Union Medical College Hospital (approval number: HS-2617) with informed consent obtained from all patients in accordance with the Declaration of Helsinki.

I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.

Data Availability

The raw clinical data were protected and not available owing to data privacy laws. The de-identified datasets supporting the findings of this study are available for academic purposes on request from the corresponding author, Qiang Sun ([email protected]), for five years, with the approval of the Institutional Ethical Committees.

View the discussion thread.

Thank you for your interest in spreading the word about medRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Citation Manager Formats

• EndNote (tagged)
• EndNote 8 (xml)
• RefWorks Tagged
• Ref Manager
• Tweet Widget
• Facebook Like
• Google Plus One

Subject Area

• Addiction Medicine (342)
• Allergy and Immunology (665)
• Anesthesia (180)
• Cardiovascular Medicine (2625)
• Dentistry and Oral Medicine (314)
• Dermatology (222)
• Emergency Medicine (397)
• Endocrinology (including Diabetes Mellitus and Metabolic Disease) (930)
• Epidemiology (12175)
• Forensic Medicine (10)
• Gastroenterology (756)
• Genetic and Genomic Medicine (4064)
• Geriatric Medicine (384)
• Health Economics (676)
• Health Informatics (2625)
• Health Policy (997)
• Health Systems and Quality Improvement (979)
• Hematology (360)
• HIV/AIDS (845)
• Infectious Diseases (except HIV/AIDS) (13659)
• Intensive Care and Critical Care Medicine (790)
• Medical Education (398)
• Medical Ethics (109)
• Nephrology (430)
• Neurology (3832)
• Nursing (209)
• Nutrition (570)
• Obstetrics and Gynecology (734)
• Occupational and Environmental Health (690)
• Oncology (2008)
• Ophthalmology (581)
• Orthopedics (238)
• Otolaryngology (304)
• Pain Medicine (250)
• Palliative Medicine (73)
• Pathology (471)
• Pediatrics (1107)
• Pharmacology and Therapeutics (459)
• Primary Care Research (447)
• Psychiatry and Clinical Psychology (3400)
• Public and Global Health (6499)
• Radiology and Imaging (1390)
• Rehabilitation Medicine and Physical Therapy (806)
• Respiratory Medicine (869)
• Rheumatology (400)
• Sexual and Reproductive Health (407)
• Sports Medicine (338)
• Surgery (441)
• Toxicology (52)
• Transplantation (185)
• Urology (165)