literature review on quality control pdf

Agency Use of Quality Control Plans for Administering Quality Assurance Specifications (2022)

Chapter: chapter 2 - literature review.

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

10 Introduction This chapter documents the important findings from the literature review of agency use of QCPs for administering QA specifications. The purpose of this chapter is to establish the back- ground and context for the findings from the survey and case examples presented in Chapters 3 and 4, respectively. A discussion of QA programs and QA approaches to alternative-project- delivery methods, including DB, CM/GC, and P3, provides a framework for understanding the requirements of contractors regarding QCPs for various project-delivery methods. The chapter then provides a brief discussion on the use of a contractor’s quality data in the acceptance deci- sion. Construction quality and inspection are then presented to provide the background for DOT compliance monitoring and inspection of contractor implementation of QCPs (see FHWA 2004a). Next, the chapter discusses the overview of contractor QCPs, types of QCPs, requirements of QCPs, industry guidance of developing QCPs, and review and acceptance of QCPs. The chapter con- cludes with a detailed discussion on the various QC requirements used by state DOTs and how DOTs use QCPs for administering QA specification. Overview of Quality Assurance Programs QA programs are developed and implemented by state DOTs and federal transportation agencies across the country. All federal-aid projects on the NHS are required to comply with 23 CFR 637. For projects off the NHS, the established QA procedures that are approved by state DOTs and satisfy the federal requirements can be used for material acceptance (FHWA 2016). The development of QA programs is an evolutionary process, and the form and elements of QA programs vary among state DOTs. However, a good QA program often shares a similar feature that “seeks to balance the cost of testing and inspection for a project with the materials quality and performance risk to maximize the benefit to the traveling public” (Transportation Research E-Circular E-C249 2019). In the past two decades, the use of alternative-project-delivery methods (e.g., DB, CM/GC, and P3) challenged the traditional quality-management approach where the contractor per- formed QC while the state DOT conducted QA. The roles of state DOTs and contractors in quality-management systems are changing (Molenaar et al. 2015b). For example, under the tradi- tional DBB delivery method or a CM/GC agreement, the agency-dominated system is mainly responsible for quality management. Under DB or P3 agreements, the responsibility for quality management is shared to varying degrees between the contractor and the state DOT (Caltrans 2015; FHWA 2012). In today’s work environment, QA is viewed as an umbrella under which quality-management activities take place. Use of QA is now favored over the previously used abbreviation QA/QC or QC/QA (Caltrans 2015). C H A P T E R 2 Literature Review

Literature Review 11   In accordance with 23 CFR 637, a comprehensive construction QA program includes the following six core elements: (1) contractor QC; (2) agency acceptance; (3) IA; (4) dispute resolu- tion; (5) personnel qualification; and (6) laboratory accreditation/qualification. These six core elements of QA apply regardless of the project-delivery method (FHWA 2012). The following section briefly discusses these core elements related to QC. • Contractor QC: According to the Transportation Research E-Circular E-C249, agencies are encouraged to require contractors to perform QC sampling and testing that may result in reducing variability and increasing long-term performance (2019). If a con- tractor’s QC test results are part of the acceptance decision, they must be taken inde- pendently of the agency verification sampling. The verification test results are sampled with control by the state DOT, independently tested, and independently compared to the con- tractor’s independent QC test results (FHWA 2004b; Transportation Research E-Circular E-C249 2019). • Agency Acceptance: The purpose of acceptance is to evaluate the quality of the product (e.g., the degree of compliance with contract requirements) and establish payment when appro priate. There are several factors involved in the acceptance process, including agency responsibilities; quality measure; validation of QC data when contractor test results are used; inspection; and risks to the agency and the contractor (Hughes 2005; FHWA 2012). The Transportation Research E-Circular E-C249 highlighted that there is “a need for additional checks and balances within a QA program when contractor’s tests are included in the accep- tance decision since a small portion of contractors have taken advantage of weaknesses in state QA programs to increase their profit by reducing their costs and/or increasing pay adjust- ments using falsified test results” (2019). • IA: The purpose of IA is to ensure the reliability of all data used in the agency’s acceptance decision—including both the agency’s verification test results and the contractor’s QC testing (FHWA 2016). • Dispute Resolution: 23 CFR 637 requires agencies to have a dispute-resolution system in place to resolve possible discrepancies between the QC data and the agency’s acceptance data if QC testing data will be included in the agency acceptance. • Personnel Qualification: 23 CFR 637 requires that all personnel performing sampling and testing for QC used in the acceptance decision, verification, or IA are required to be quali- fied. FHWA also noted that agencies participate in state, regional, or national technician qualification or certification programs to ensure that technicians and inspectors are properly qualified (2012). • Laboratory Accreditation/Qualification: 23 CFR 637 requires that all contractor, vendor, and state DOT testing used in the acceptance decision must be performed by qualified laboratories. Historically, agencies used the term QC/QA with QC referring to the contractor’s role and QA to the agency’s role. This term implied that QC and QA are separate functions; in fact, QA refers to the overall system for assuring project quality, with QC being one element of a comprehensive QA program. Therefore, the transportation industry has moved away from the term QC/QA and now uses QA (FHWA 2012).

12 Agency Use of Quality Control Plans for Administering Quality Assurance Specifications It is important to note that QA involves “continued evaluation of the activities of plan- ning, design, development of plans and specifications, advertising and awarding of contracts, construction, maintenance, and the interactions of these activities” (Transportation Research E-Circular E-C235 2018). The recent research found that “current material QA practices vary widely among DOTs, and what is acceptable for one DOT may not be for another” (Scott and Molenaar 2017). The study also highlighted that current “QA practices are also evolving, par- ticularly with regard to increased use of alternative project delivery methods that shift more responsibility to industry for managing quality, and with advances in general understanding of materials/product behavior and use of more performance-based quality measures and non- destructive testing technologies that provide for continuous sampling and data collection” (Scott and Molenaar 2017). Use of Contractor’s Quality Data in the Acceptance Decision In 1995, 23 CFR 637 was revised to expressly allow the use of contractor QC test results in a DOT’s acceptance decision and allows the use of consultants in the IA program and for verifi- cation sampling and testing. When quality is not being maintained, state DOTs can reallocate resources to increase the frequency of QA testing and verification to improve quality and lessen risk. FHWA T6120.3 noted that QC sampling and testing results may be used as part of the acceptance decision provided that: • The sampling and testing are performed by qualified laboratories using qualified sampling and testing personnel. • The QC sampling and testing is evaluated under an IA program. • The quality of the material is validated by the verification sampling and testing. The verifica- tion testing is performed on samples that are taken independently of the QC samples. The verification sampling and testing are performed by qualified testing personnel employed by the DOT or its designated agent (i.e., consultant under direct contract with the DOT). Use of a third-party testing-and-inspection firm hired by the contractor does not relieve the agency of its responsibility for verification. • The DOT has a dispute-resolution system in place to resolve possible discrepancies between the contractor’s QC and the agency’s verification data (FHWA 2004b). Additional guidance on the CFR requirements provided through Federal-Aid Policy Guide Non-Regulatory Supplement NS 23 CFR, Part 637B, issued in 2006, clarified the following key points: • A DOT’s acceptance program “should provide a reasonable level of inspection to adequately assess the specific attributes which reflect the quality of the finished product” (23 CFR 637B). Inspection should cover component materials at the time of placement or installation as well as the workmanship and quality of the finished product. The intended function of each part of QA is important because each function should supplement the other. The analogy has been used of QA being similar to a three-legged stool, with one leg being QC, one leg being acceptance, and the third leg being IA. With any leg missing, the whole is unbalanced. The present-day concept of QA is that QC is the responsibility of the contractor, acceptance is the responsibility of the agency (although this responsibility may involve contractor test results), and IA is conducted by an independent third party (Hughes 2005).

Literature Review 13   • Sampling and testing frequencies may vary across state DOTs because the quality and uni- formity of materials vary. State DOTs have some discretion to adjust testing frequencies for materials with a “history of accurate, uniform test results that consistently meet specifica- tion requirements. The rate of testing should be higher on newly developed material sources, sources with questionable quality, sources with a wide range of test results, and sources with failing test results” (23 CFR 637B). • If a contractor’s quality data is used in the acceptance decision and the DOT and contractor test results do not compare, the frequency of the DOT’s verification testing should be increased. • The DOT should obtain the contractor’s test data immediately, no later than 24 hours after sampling is completed. The DOT’s test results should not be given to the contractor until after the contractor results are received. • Test results should not be discarded unless the sampling or testing is flawed. Additional tests may need to be conducted when the quality of the material is in question. If addi- tional tests are performed, the acceptance and pay criteria need to be adjusted to account for the additional test results. • All test procedures used in the acceptance decision should be in the scope of accreditation for the DOT central laboratory. It is important to take the time to understand the strengths and weaknesses of specifications and test procedures when writing them. In addition, it is key to consider the quality assurance specifications and procedures from not just the agency’s perspective. Whenever possible, it is best practice to obtain all relevant data, including the originally measured weights. Doing so would afford an excellent opportunity to perform a data forensic investigation (Transportation Research E-Circular E-C249 2019). Quality Assurance for Alternative-Contracting Projects Alternative-delivery methods, including DB, CM/GC, and P3, are increasingly used among state DOTs. The growing use of these alternative-delivery methods has created a new environment for state DOTs to manage the quality of transportation projects. NCHRP Report 808: Guidebook on Alternative Quality Management Systems for Highway Construction pointed out that “when using project delivery methods in which the contractor is selected before the design is complete and is expected to contribute to the design, the agency should consider the impact of that shift on quality management planning and execution at every phase of project development” (Molenaar et al. 2015a). Figure 1 shows a generic QA framework that includes typical project quality activi- ties (e.g., construction QC, construction acceptance, and IA) associated with the roles and respon- sibilities of owners, constructors, and designers. A dashed line is used to separate responsibilities of the agency, contractor, and designer. For example, items above the dashed line are the respon- sibility of the agency. It is noted that design quality is typically not included in traditional DBB projects but plays an important role for alternative-contracting projects (Molenaar et al. 2015a). The following sections briefly discuss QA for DB, CM/GC, and P3 delivery methods. Overview of QA for DB Projects One of the main characteristics of DB project delivery is that the design and construction are performed under one contract by the same entity (i.e., the design-builder). This single source of responsibility for both design and construction in DB projects is the key difference to quality management between DB and DBB projects. NCHRP Synthesis 376: Quality Assurance in

14 Agency Use of Quality Control Plans for Administering Quality Assurance Specifications Design-Build Projects highlighted that “with the changing quality roles found in the DB delivery method, it is imperative that quality responsibilities and the responsible parties are clearly stated in the contract documents” (Gransberg et al. 2008). FHWA also indicated that the “owner must define the responsibilities of the design-builder and the contracting agency” when preparing the request for proposal (RFP) (2012). Figure 1. Generic QA model (Source: Molenaar et al. 2015a). The management of quality in the DB project is of utmost importance and requires that a department of transportation (DOT) contemplating the use of a DB project delivery prepare a thorough and thoughtful approach to communicating the DB project’s quality requirements as well as the administrative and technical mechanisms that the DOT intends to use to manage both design and construction quality (Gransberg et al. 2008). Several state DOTs have developed QA guidance for their DB projects. For example, Texas DOT (TxDOT) has used its QA program to provide statewide consistency and a programmatic approach to QA for DB projects where the design-builder (i.e., DB contractor) test results are used in the acceptance decision regardless of how the project is funded (TxDOT 2017). Figure 2 illustrates the typical components and the relationships between the parties for the QA program in TxDOT. TxDOT noted that the design-builder in DB projects establishes a systematic QCP to define processes, methods, and documentation for delivery of QC. The plan clearly defines the authority and responsibility for the administration of QC to ensure that the work is delivered in accor- dance with the contract documents (TxDOT 2017).

Literature Review 15   Recently, Colorado DOT (CDOT) has developed a QA program for its DB projects. CDOT’s QA program consists of a QC program, an owner-acceptance program, and an IA program (CDOT 2020). Figure  3 presents the relationship among these three main components of CDOT’s QA program. CDOT noted that project management and quality management require some adjustment to address the shift in responsibility from the owner to the design-builder, but “it should be under- stood that the fundamental principles of quality assurance do not go away with the DB contract- ing method” (CDOT 2020). Figure 3 shows that CDOT’s QA program for DB projects allows for the use of the contractor’s performed test results, referred to as independent contractor quality control (ICQC), as part of an acceptance decision if those results are validated by the Owner Verification Testing (OVT) results performed by a representative for CDOT (CDOT 2020). Figure 2. TxDOT QA program for DB projects (Source: TxDOT 2017). The D-B project delivery system offers several documented benefits over the traditional D-B-B method on certain projects. While D-B offers the design-builder more control over design, materials, and construction methods than D-B-B, the agency still has an important role in assuring quality. As agencies develop DB procurement documents, it is important that roles and responsibilities for design- builder QC and agency acceptance be clearly defined. The responsibility for acceptance by the agency (or their designated agent) is applicable regardless of the project delivery method used (FHWA 2012). Figure 3. CDOT program for DB projects (Source: CDOT 2020).

16 Agency Use of Quality Control Plans for Administering Quality Assurance Specifications NCHRP Report 808 indicated there are several QA models for DB projects, including assurance QA, variable QA, and oversight QA models (Molenaar et al. 2015a). Kraft and Molenaar found that the oversight QA model is the most feasible for DB projects (2015). In the oversight QA model, the agency’s role is to ensure that both the designer and contractor QA plans are effective at meet- ing the agency’s quality requirements and that the plans are being implemented (Kraft and Molenaar 2015). Figure 4 shows the oversight QA model for DB projects. Figure 4 also shows that the design-builder is responsible for all QC and acceptance for the project (Molenaar et al. 2015a). Overview of QA for CM/GC Projects Through an analysis of federal projects, Uhlik and Eller showed that CM/GC project delivery provides two main features that help enhance the project quality: (1) a system of checks and bal- ances exists between design and construction, and (2) input on quality is provided during design by a construction expert (1999). The FHWA Every Day Counts (EDC-2) indicated that one of the main benefits of CM/GC project delivery is improved design quality. Specifically, the FHWA EDC-2 highlighted that under the CM/GC delivery method “the contractor is able to review the designs and provide feedback, answer designer questions and provide changes. By including the contractor review, the designer can produce better designs that reduce issues in con- struction and prevent change orders that can lead to project overruns” (2014). The construction quality management of CM/GC projects does not differ greatly from tradi- tional DBB projects because the owner still occupies the same contractual position with respect to the designer and builder (Molenaar et al. 2015a). Several state DOTs have directly applied their Figure 4. Typical QA model for DB projects (Adapted from Molenaar et al. 2015a).

Literature Review 17   QA program for DBB projects to CM/GC projects with little alteration. For example, California DOT (Caltrans) developed its construction QA program manual that is applied to both DBB and CM/GC projects (Caltrans 2015). The Caltrans manual highlighted the following: The role of the Construction QA program is to provide confidence that the quality of the materials and workmanship incorporated into highway construction projects conforms to the requirements of the plans and specifications. The provisions of this manual apply to DBB projects and projects under a CM/GC agreement (Caltrans 2015). Similarly, CDOT’s Construction Manager/General Contractor Manual summarized the roles of the QA program as following: [A QA program] for CM/GC projects is performed as it would be for a DBB project. Construction Management, Testing and Inspection will be through CDOT or a Consultant Project Engineer and staff. The Contractor will have developed a Quality Control Plan during the Preconstruction Phase that should be referenced during the Construction Phase (CDOT 2015). Overview of QA for P3 Projects Public-Private Partnership (P3) Procurement: A Guide for Public Owners concluded that the traditional means of QA is not an option for P3 projects (FHWA 2019). Most transportation P3 projects in the United States have used a best-value procurement approach. QA and QC organization and approach and the quality of past performance are among the key technical evaluation factors for P3 best-value selection. FHWA highlighted that “the agency must focus on the quality of the proposer teams and their proposals, and rely on the P3 structure itself and the contract terms and conditions to give the P3 contractor incentives to provide a high-quality project and perform operations in a way that meets public needs” (2019). In P3 projects, the concessionaire often has a significant role in QA and QC. The Virginia DOT (VDOT) requires the QA plan to be separated and distinct from the QCP for both design and construction (VDOT 2018b). The concessionaire in P3 projects prepares the Quality Man- agement System Plan (QMSP) that includes an acceptable QA plan and an acceptable QCP. The QMSP defines a uniform process approach to design and construction quality management, quality procedures, records keeping, and document management/control that the concession- aire shall adhere to throughout the duration of the project (VDOT 2018b). The minimum requirement for the design QMSP includes the following: • Written documentation and definition of the project’s design criteria, standards, and processes; • Procedures for the performance of experienced senior engineers’ detailed checks of all design reports, calculations, drawings, and specifications; • Directions for interdisciplinary reviews by technical and management staff to provide coor- dination and uniformity among section designs; • Procedures for constructability reviews to facilitate the timely planning of construction activities; • Procedures for maintainability reviews to ensure feasibility of future maintenance and opera- tion; and • QA audit checklists (VDOT 2018b). The minimum requirements for the construction QMSP include the following: • Staffing plan; • Inspection plan; • Testing plan; and • Construction inspection checklists (VDOT 2018b). Kraft and Molenaar found that the acceptance QA model is the most feasible for P3 projects (2015). In the acceptance QA model, the agency is responsible only for verification testing and

18 Agency Use of Quality Control Plans for Administering Quality Assurance Specifications final acceptance, and the concessionaire is responsible for all other quality roles and responsi- bilities. Figure 5 shows the acceptance QA model for P3 projects. The acceptance QA officer provides the agency with the least amount of direct control over QA of a project. The main agency’s focus in the acceptance QA model is to perform oversight of the design and construc- tion quality-management efforts to satisfy the legal responsibilities. As long as the QCPs meet the requirements of the contract, the agency approves them (Molenaar et al. 2015a). FHWA summarizes the difference in QA between P3 and traditional DBB projects as follows. Figure 5. Typical QA model for P3 projects (Source: Molenaar et al. 2015a). In P3 projects, the concessionaire is responsible for design and construction of the project as well as operations and maintenance over a long-term period. Furthermore, the concessionaire is obligated to hand the project back to the public owner at the end of the term, at a pre-defined level of service and quality. Consequently, the concessionaire has greater financial incentive to make investment decisions that are optimized over the life of the project rather than the incentives contractors have with traditional D-B-B delivery methods (FHWA 2019).

Literature Review 19   Overview of Construction Quality Control Plans A QCP is an essential component of managing quality. Because the QC function is the respon- sibility of the contractor, it is critical to the success of a highway-construction project that the contractor has a functional and responsive QCP. The Federal Lands Highway (FLH) Construction Manual emphasized that an effective QCP does not depend on the oversight and interaction of agency inspectors to control quality (FLH 2009). There is no one-size-fits-all approach to devel- oping a QCP. Each contractor has unique means, methods, and strategies to achieve quality that meets the agency’s needs and expectations. However, FLH highlighted that “a QC plan is not a vague feel-good commitment to quality and contract compliance. It is a detailed account of how the people and processes of an individual contractor will function to deliver quality” (2009). The main purpose of the QCP is to provide a framework or procedures that clearly describe how the contractor intends to address various quality requirements, given the project requirements and contractor resources, preferences, and risk tolerance. The QCP is developed to measure the quality characteristics and inspect construction activities when corrective actions can be taken to prevent nonconforming materials from being incorporated into the project (Burati et al. 2003). As a result, determining quality characteristics to measure for QC in a QC plan is an important step. Table 1 summarizes the typical quality characteristics for hot-mix asphalt (HMA) and portland cement concrete pavement (PCCP) as an example of good quality characteristics included in a QC plan. There are several approaches to implementing a QCP. For example, the QCP can be stipu- lated by the state DOT, chosen by the contractor, or a combination of both. As mentioned previ- ously, there is no one-size-fits-all QCP. Each approach has advantages and disadvantages. For example, if a DOT specifies the minimum requirements for a QCP, the contractor may view these requirements as all necessary for developing the QCP. On the other hand, if a DOT specifies all the requirements and properties for a QCP, the contractor may view the QCP as the agency’s plan rather than the contractor’s plan, which may result in less contractor innovation. FHWA noted that the approach selected to establish the QC plan will have a substantial impact on “the way that the QC plan is implemented and the decisions that must be made throughout the implementation of the plan” (2003). Further, it is important to reevaluate each quality characteristic selected for QCPs after QC data is collected, analyzed, and reviewed by the agency to determine whether to include the quality characteristic for QC purposes or for acceptance testing (Burati et al. 2003). HMA PCCP • Aggregate quality, including fractured faces, sand equivalency, cleanliness, etc.; • Nuclear density; • Gradation of critical sieve sizes; • Plant and discharge temperatures; • Degree of aggregate coating; and • Moisture content of fine aggregate and/or of finished mix. • Aggregate quality; • Gradation of critical sieve sizes; • Air content; • Water-to-cementitious-materials ratio; • Mix temperature; • Slump; and • Spread (for self-consolidating concrete). Table 1. Typical quality characteristics for QC testing (Burati et al. 2003). Quality and the processes that deliver quality are a somewhat open-ended concept. No matter how much detail is in the plan, it can always be argued that more could be, or should be, included (FLH 2009).

20 Agency Use of Quality Control Plans for Administering Quality Assurance Specifications Types of Quality Control Plans Typically, there are two types of QC plans: (1) generic QCP and (2) contractor-specific QCP. NCHRP Synthesis 346: State Construction Quality Assurance Programs highlighted that the QC plan should be established based on a specific contractor or operation, but many state DOTs choose to develop a generic plan for all contractors (Hughes 2005). The following sections briefly discuss these two types of QCPs. Generic QCPs The purpose of using a generic QCP is to create a common approach for all contractors. State DOTs may use a generic QCP because of an inadequate contractor QC staff, contractor competition, and QC cost. Under a generic QC plan, state DOTs often stipulate minimum QC plan procedures (Burati et al. 2003). There are several potential drawbacks of using a generic QCP for both contractors and state DOTs. For example, the sampling and testing procedures and frequency stipulated in a generic QC plan are typically not operation-specific, and the con- tractor may face an out-of-control product (Burati et al. 2003). Likewise, state DOTs face chal- lenging issues when encountering a contractor that needs more frequent sampling and testing to maintain control. If state DOTs use a generic QCP, FHWA highlighted the following key items related to QC: • If state DOTs stipulate QC requirements, then these requirements will have to be generic in nature to apply to different contractors and different plant operations. • If state DOTs decide to establish initial statewide action or control limits, they must evaluate data from a number of typical operations. In addition, because the control limits are not devel- oped for each specific plan or operation, state DOTs should transfer the responsibility related to the control limits to the contractors quickly to establish specific limits for each operation. • To establish a generic QCP, state DOTs must collect data from a number of typical opera- tions based on their historical records, and data should be randomly sampled. If there is not sufficient data, the needed data must be obtained (Burati et al. 2003). Operation-Specific QCPs The operation-specific QCP is often considered as an ideal approach because of varying QC procedures for different construction operations. Typically, construction operations with a history of QC problems (e.g., high-risk items) will require more frequent sampling and testing than operations with few problems (e.g., low-risk items). FHWA emphasized that the contractor is in the best position to know the types of tests and the testing frequencies that are the most suitable indicators of QC (2003). Under the operation-specific QCP, the data used for estab- lishing the plan should come from prior production of similar products representative of that operation. The contractor will be responsible for gathering the data and establishing the operation-specific action or control limits. If state DOTs use an operation-specific QCP, FHWA highlighted the following key items related to QC: • To establish the action limits for control, a QC plan is developed based on the contractor’s specific plant operation and a measure of the average, and the variability must be examined over a period of time. • The data used to develop an operation-specific QCP should be collected in the same manner as that in the QCP that will be implemented. • To establish QC tests and frequencies, the selection of testing procedures and frequencies typically relies on each individual contractor. One of the keys to reaching a balance in testing

Literature Review 21   frequency is to compare the testing frequency to the rate and consistency of production. For example, if the production tends to be consistent, then less frequent testing is allowed. If there are many interruptions, then more frequent testing is required (Burati et al. 2003). Establishment of Quality Control Plans A contractor is responsible for developing a QCP. However, state DOTs need to understand the QCP development process to provide general guidance to the contractor on required plan elements. The development process of a QCP depends on several factors, including the project components required for a QCP; project-delivery methods used; and the relationships among the general contractor, subcontractors, and material suppliers. Typically, a QCP includes the following items: • Providing relationships among parties involved in the project or the work, including contact information for key personnel; • Listing personnel, along with their certifications and relevant experience to the project or the work; • Describing roles and responsibilities of each person involved in the QC process; • Providing required documentation for materials and manufactured products; • Providing references (e.g., test performed, test method, sampling plan, or frequency) for each material or component associated with specification provisions; • Describing monitoring activities along with frequency and other relevant information; • Providing a procedure for evaluating data, including analysis methods and tools; • Describing the means for maintaining control of the work, including the approach to adapt processes or stop work until it complies with expectations and re-equipments; • Describing corrective-action plans to address out-of-control processes and remedy deficient work; and • Describing the process of documentation of the work supporting the QCP (Cavalline et al. 2021). Appendix D provides a QCP outline for concrete-paving projects. For a given project, it is helpful if the state DOTs provide input on the content and level of detail required for an acceptable QCP, unless the contractor already has a documented company QCP. The Standard Specifications for the Construction of Roads and Bridges on Federal Highway Projects (FP-14) indicates that a QCP addresses all contract work categories that include related work items performed in a single operation. For example, a grading and drainage category of work may include the following work items: clearing, excavation, embankment, drainage, and slope protection. Depending on the nature of the project and the organization, the contract work categories may vary. According to FHWA, the typical construction categories are: • Grading and drainage; • Structures; • Masonry; • Pavement structures; • Safety appurtenances; • Seeding and landscaping; • Permanent traffic control; and • Temporary traffic control (2004a). Several state DOTs—California, Kansas, and Utah—indicated that the minimum requirements for a QCP include: • QCP certification by the QC manager (QCM); • QC organization;

22 Agency Use of Quality Control Plans for Administering Quality Assurance Specifications • QCP distribution list; • QC inspection plans; • Sampling and testing plan; • Random sampling plan; • Laboratories and equipment; • Action limits and corrective-action plans; and • QC documents. FP-14 requires that a QC plan be submitted at least 14  days before the start of work. No work on a construction category will be performed until the QC for that category is accepted. In addition, FP-14 stated that approval of a QCP does not imply that the plan will result in contract compliance. When changes occur in the contract, work progress, or personnel, the QCP should be revised. For each construction category, FP-14 specified three main areas in a QCP: (1) QC personnel; (2) QC procedures; and (3) records and documentation. The following sections discuss briefly these three areas. QC Personnel QC personnel in a QCP may vary from agency to agency. Typically, the QC personnel include the QCM, inspectors, testers (e.g., laboratory technicians and QC sampler), and other personnel directly involved with inspection and testing. Figure 6 shows the relationship of the QCM, QC inspector, laboratory technician, and QC sampler. The main responsibilities of a QCM include: • Directing a program; • Providing and implementing a QCP; • Reviewing test results; • Reviewing inspection reports, material certificates, and construction-process records; • Coordinating QC activities; and • Other responsibilities (Hancher et al. 2002). For a full-time, on-site QCM, FP-14 requires that a QCM has no responsibilities for per- forming testing and inspection, managing the project, or performing operations other than managing QC. For a part-time, on-site QCM, FP-14 requires that a QCM has at least two years’ experience in highway construction, inspection, QC, and material testing. Additionally, FP-14 states that the required qualifications of a full-time, on-site QCM are: • One year experience managing QC on construction projects of similar type and scope; and • One of the following: – Two years’ experience as a construction project manager or superintendent on construc- tion projects of similar type and scope; – Three years’ experience as a project engineer, resident engineer, foreman, construction inspector, or equivalent on construction projects of similar type and scope; or – National Institute for Certification in Engineering Technologies (NICET) Level III certifi- cation or equivalent in highway construction or highway material. Figure 6. Contractor QC personnel (Source: Hancher et al. 2002).

Literature Review 23   The main responsibilities of a QC inspector include: • Inspecting source material; • Inspecting plant operation; • Inspecting on-site construction; and • Recording inspection results (Hancher et al. 2002). The main responsibilities of a laboratory technician include: • Calibrating testing equipment; • Performing QC testing; and • Reporting test results (Hancher et al. 2002). The main responsibilities of a QC sampler include: • Deciding QC sampling place; • Taking QC samples; • Taking acceptance samples; and • Recording sample places (Hancher et al. 2002). A QC inspector is required to have at least 2 years’ experience inspecting projects of similar complexity and with training related to the work to be inspected (FP-14). A tester is required to have at least one year’s experience in the type of sampling and testing required, with one of the following for the type of sampling and testing performed: • NICET Level II certification in highway material or equivalent state or industry certification; • Certification by a regional certification program, such as Western Alliance for Quality Trans- portation Construction (WAQTC), Northeast Transportation Technician Certification Program (NETTCP), Southeast Task Force for Technician Training and Qualification (STFTTQ), or Multi-Regional Training and Certification (M-TRAC); or • At least one year of employment by an AASHTO-accredited laboratory performing equiva- lent sampling and testing (FP-14). QC Procedures This portion of a QCP is challenging to develop because the process-control procedure evolves over time. For execution of work, the QC procedures describe the inspection, testing, and other activities to be performed for each phase of work, including preparatory, start-up, and production phases. Preparatory Phase FP-14 requires the contractor to hold a meeting before starting each work category. The meet- ing invitees will include the project superintendent, work foreman, contracting officer, QCM, and appropriate subcontractors, and will discuss the following topics: • Contract requirements for the work, including acceptance procedures, schedule, and con- trol strip; • Process and equipment for constructing the work; and • Plan for inspection, PC, testing, measuring, and reporting the work. Additionally, the contractor should complete the following items in the preparatory phase: • Reviewing and coordinating certifications, submittals, plans, drawings, and permits; • Verifying the capabilities of equipment, material, and personnel, and providing training as necessary; • Establishing a detailed testing schedule based on the production schedule; • Ensuring preparatory testing and inspection is accomplished; and • Reviewing accuracy of the surveying and staking.

24 Agency Use of Quality Control Plans for Administering Quality Assurance Specifications Start-Up Phase The main purpose of this phase is to inspect, test, and report start-up work according to the QCP and ensure the work conforms to the contract. FP-14 indicates that a start-up meeting including the project superintendent, inspectors, testers, contracting officer, and QCM will be held to discuss the following topics: • Reviewing the contract, the construction processes, and the inspection, testing, and reporting requirements with the personnel performing the work; and • Explaining procedures that will be followed if defective work is identified. Production Phase This phase includes the following activities: • Inspect, test, and report according to the QCP and evaluate the acceptability of the work produced; • Identify and correct deficiencies; and • Request owner inspection and acceptance. For sampling and testing, the quality characteristics, test procedures, and inspection activities related to early indications of PC should be stipulated, and decisions will depend on the type of operation for which the QCP is being developed (Burati et al. 2003). It is noted that a testing fre- quency must be established to create a balance with the number of tests performed. The state DOT and contractor may use increased testing frequencies initially and then reduce the frequency as the operation is in control. FP-14 requires that the QC procedure in a QCP list the material to be tested by: • Pay item; • Applicable requirements of the sampling, testing, and acceptance requirements tables; • Persons responsible for performing the sampling and testing; • Laboratory-testing facilities to be used for PC and project testing; and • Proposed reporting formats. For each category and phase of the operation, FLH showed that the QC plan should address the following questions (2009): • Who will be responsible for QC during the operation? • What will that person do to ensure contract compliance? • Where will these activities be performed? • When will these activities be performed? • How will inspections be performed? Records and Documentation The purpose of this portion of a QCP is to describe the reporting format for inspection, testing, certification, and daily reports. The contractor is responsible for obtaining, reviewing, and verifying certifications for work and submitting certifications when required. FP-14 indicates that contractors submit written QC and construction-operation reports according to the QCP. The reports document meetings; work locations; labor and equipment used, including actual hours worked; testing and measurement activities; inspection results; defi- ciencies observed; corrective actions taken; and process changes. Additionally, control charts are used to (1) document variability of the process, (2) identify production and equipment problems, and (3) identify actions to improve processes or quality. However, it is important to note that documentation should be the minimum necessary to demonstrate that the contractor is pro- ducing work and using materials and equipment of acceptable quality.

Literature Review 25   Review of Quality Control Plans The QCP is typically reviewed by state DOTs before any construction work can begin. The QCP is found to conform to the contract requirements or returned for corrections. Specifi- cally, FLH indicated that when evaluating a QCP, the evaluator must consider the following questions (2009): • Does the plan address each category of work? • For each category, does it address the preparatory, start-up, and production phases? • For each category and phase, does it address the who, what, where, when, and how questions presented in the QC procedures? QCPs may be evaluated as conditional acceptance for partial plans. The conditional accep- tance of a QCP often involves the plan submitted with good faith but missing numerous amounts of key information. The conditionally accepted QCP may allow work to begin with the condition that the contractor will supplement the plan with the missing information within a specific time frame. The conditional acceptance is typically appropriate if the work deficiencies in the plan are not scheduled to commence for several weeks or months (FLH 2009). The partial QCP often involves subcontractors or suppliers at later phases of construction that have not been arranged at the time the contractor is ready to begin work on the initial phases. For this type of QCP, the acceptance letter should state that the contractor may only begin work that is covered by the partial plan, and the agency will not perform QA, accept, or pay for work not covered by the plan (FLH 2009). When contractors provide the QCP with little or no useful information related to the contrac- tor’s quality process, the QCP should not be accepted. To reject the QCP, state DOTs should specify the major reasons for the rejection in the rejection letter (FLH 2009). Additionally, the work related to the rejected QCP should not begin until an acceptable plan is submitted. Sample Industry Quality Control Plans The QC plan should minimize any parroting or paraphrasing of requirements in the contract and should avoid simply promising to comply with the contract. These kinds of statements and assurances add little value. The QC plan must go beyond the contract requirements and address the contractor’s organizational process for consistently delivering those requirements (FLH 2009). Chronic deficient work is evidence that the QC plan is not effective, and improvements to the plan should be required. Likewise, when the agency is compelled to increase its inspection levels to compensate for poor contractor quality control, that also is evidence that the QC plan is not effective (FLH 2009). Industry QCPs vary substantially among construction companies. Because specific company QCPs cannot be discussed in the synthesis report, this section briefly describes three sample templates/guidance for developing and implementing QCPs: (1) construction QCP templates

26 Agency Use of Quality Control Plans for Administering Quality Assurance Specifications developed by the U.S. Army Corps of Engineers (USACE); (2) QC guidance from the National Ready Mixed Concrete Association (NRMCA); and (3) a model QCP prepared by NETTCP. USACE Construction QCP Guidance The contractor QCP, a foundation of quality work, is an outline of the planned QC proce- dures. As a result, the contractor must consider fully the specific contract requirements and special factors for a given project when establishing a QC plan. It is critical that the QCM is the author or coauthor of the QCP to ensure that all quality requirements contained in the contract are included and that the QCM is thoroughly familiar with the plan (USACE 2015). The USACE requires that a contractor QCP be developed and submitted. The USACE pro- vides two templates for a QCP: traditional DBB-construction and DB-construction QC templates. The QCP template for DB projects has additional components that are not required to be contained within a QCP for DBB projects. Both QCP templates describe the process for creating an effective, efficient, and auditable quality-management system that includes the process and procedure to perform QC tasks of activities in all phases of a project. Table 2 summarizes the outline of QCPs for both DBB and DB projects. The detailed instructions of each section can be found in the USACE templates (USACE 2021). NRMCA Guidelines for Quality Manual for Ready-Mixed Concrete Companies NRMCA is a leading industry advocate working to expand and improve the ready-mixed con- crete industry. In 2008, NRMCA approved the guidelines for a quality manual for ready-mixed concrete companies. The guideline provides the structured development of a quality manual that documents the process to ensure quality of the company’s products and services (NRMCA 2008). NRMCA noted that the term QC plan is used within the context of the quality manual as the overview document to describe the company quality-management system. QC plan template for DBB projects QC plan template for DB projects • Master Table of Contents • Acronyms and Abbreviations • Section A: Quality Control Organization • Section B: Project Staff Qualifications • Section C: Appointment Letters • Section D: Submittal Procedures • Section E: Testing Procedures • Section F: Procedures for Performing the Three Phases of Quality Control • Section G: Procedures for Tracking Construction Deficiencies • Section H: Reporting Procedures • Section I: Definable Features of Work • Master Table of Contents • Contract QC Plan Cross-Reference Table • Acronyms and Abbreviations • Section A: Quality Control Organization • Section B: Project Staff Qualifications • Section C: Duties and Responsibilities • Section D: Appointment Letter • Section E: Definable Features of Work • Section F: Submittal Procedures • Section G: Testing Procedures • Section H: Deficiencies • Section I: Three Phases of Quality Control • Section J: Documentation • Section K: Completion Inspection Procedures • Section L: Training and Certification Logs Table 2. Outline of USACE QCPs for DBB and DB projects (USACE 2021).

Literature Review 27   The NRMCA guideline was developed based on 15 sections in accordance with the primary outlines in ISO 9000 certification for quality systems, along with appendices. Table 3 summa- rizes the outline of the NRMCA guideline quality manual for ready-mixed concrete companies. The detailed instructions of each section, along with the sample quality manual, can be found at the NRMCA (2008). NRMCA noted that this guideline serves as a minimum standard to support the qualifications and credibility for ready-mixed concrete companies. In addition, the guideline can be used as “a good outline for all ready mixed concrete companies desiring to progress their individual quality initiatives and establish a written quality plan endorsed and supported by the executive management with defined responsibilities for those defined initiatives” (NRMCA 2008). NETTCP Model QCP The model QCP prepared by NETTCP provides a generic tool that includes a recommended format of 10 separate sections, plus appendices, to help contractors develop complete and useful QCPs. Each section addresses major QCP items. The 10 sections of the NETTCP model QCP include: • Terms and Definitions (optional); • Scope and Applicable Specifications; • Quality Control Organization; • Quality Control Laboratories; • Materials Control; • Quality Control Sampling and Testing; • Production Facilities; • Field Operations; • Acceptance of Work (optional); • Other Relevant Contractor QC Plans; and • Appendices. NETTCP noted that the level of detailed information in any QCP varies depending on the size and complexity of the project (NETTCP 2009). NETTCP provided an example of a completed earthwork QC plan for a fictitious construction project. A detailed discussion of this completed earthwork QC plan is available at https://www.nettcp.com/ or in Appendix B, “Quality Assurance Resource,” of FHWA (2004b). Based on the NETTCP model QCP, Cavalline et al. recently developed a model QCP for a fictitious concrete-paving project, which includes • Introduction • Quality Management System • Management Responsibility • Customer Focus • Human Resources • Facilities, Plant(s), and Equipment • Materials Management • Purchasing • Order Processing and Dispatching Procedures • Concrete Production • Concrete Testing • Concrete Delivery and Site Control • Concrete Troubleshooting • Measurement, Analysis, and Improvement • Concrete Mixture Development or Selection • Appendices Table 3. Outline of NRMCA guidelines for ready-mixed concrete companies.

28 Agency Use of Quality Control Plans for Administering Quality Assurance Specifications 3 divisions: Earthwork (Division 2), Concrete Pavements and Shoulders (Division 7) and Materials (Division 10) (2021). It is noted that the information presented in the model QCP needs to be modified to meet the needs of the project, the requirements of the agency, and the preferences of the contractor. A detailed discussion of the model QCP for a fictitious concrete- paving project can be found in Appendix D, “Model QC Plan,” from Cavalline et al. (2021). Summary The literature review results presented in this chapter document the most relevant topics of QCPs and QA programs. It provides key information for understanding the state of prac- tice of QCPs used by state DOTs for administering QA specifications. The key concepts of QA approaches to various project-delivery methods, requirements of contractors regarding QCPs, construction quality and inspection, the review and acceptance process of QCPs, and the use of contractor’s quality data in the acceptance decision are discussed in detail. The literature review results presented in this chapter set the basis for the survey and case example protocols applied in this synthesis. This chapter found that the agency-dominated system is mainly responsible for quality man- agement for traditional DBB or CM/GC projects. Under DB or P3 agreements, the responsibility for quality management is shared to varying degrees between the contractor and the state DOT. However, the six core elements of a QA program—contractor QC, agency acceptance, IA, dispute resolution, personnel qualification, and laboratory accreditation—apply regardless of project- delivery methods. The purpose of QCPs is to provide a framework or procedures to measure the quality characteristics and inspect construction activities when corrective actions can be taken to prevent nonconforming materials from being incorporated into the project. Although many components of QC can be included in the contract, there is no one-size-fits-all approach to developing QCPs. A QCP describes how the personnel and processes of an individual con- tractor will function to provide PC for delivering the required quality. An effective QCP does not depend on the oversight and interaction of agency inspectors to control quality. The QCP is typically reviewed by state DOTs before any construction work can begin. The review process often involves evaluating the contractor plan associated with each category of work, along with preparatory, start-up, and production phases. The QCP should avoid paraphrasing of require- ments or simply promising to comply with the contract. Finally, the use of contractor’s data in the agency acceptance decision requires a higher level of oversight of the QCPs related to QC sampling and testing.

For years, state departments of transportation (DOTs) were solely responsible for quality control (QC) and performed testing and inspection to verify construction quality. However, since the 1970s, many state DOTs have transferred the responsibility for QC processes of their highway-construction projects to contractors.

The TRB National Cooperative Highway Research Program's NCHRP Synthesis 590: Agency Use of Quality Control Plans for Administering Quality Assurance Specifications identifies and documents the various QC requirements used by state DOTs.

READ FREE ONLINE

Welcome to OpenBook!

You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

Do you want to take a quick tour of the OpenBook's features?

Show this book's table of contents , where you can jump to any chapter by name.

...or use these buttons to go back to the previous chapter or skip to the next one.

Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

To search the entire text of this book, type in your search term here and press Enter .

Share a link to this book page on your preferred social network or via email.

View our suggested citation for this chapter.

Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

Get Email Updates

Do you enjoy reading reports from the Academies online for free ? Sign up for email notifications and we'll let you know about new publications in your areas of interest when they're released.

To read this content please select one of the options below:

Please note you do not have access to teaching notes, a literature review on total quality management (models, frameworks, and tools and techniques) in higher education.

The TQM Journal

ISSN : 1754-2731

Article publication date: 7 September 2021

Issue publication date: 24 November 2022

The purpose of this paper is to analyze the evolution of total quality management (TQM) models, frameworks, and tools and techniques in higher education (HE) over the last thirty years from 1991 till 2020, based on a literature review

Design/methodology/approach

30 articles from 52 journals were used to perform this detailed literature review. For the detailed analysis, the focus was only on articles related to TQM in higher education and specifically related to models, frameworks and tools and techniques. The study has investigated the growth of research articles, research streams, research methodologies, models and frameworks in the higher education sector and tools and techniques related to those.

This review addresses the progress and gaps in the application of TQM in HE, including the shift in global research in this area from the USA and Europe to Asia in recent years. The articles have been classified into four research methodologies and two research streams which have been reviewed in detail. The findings include reasons for multiple models/frameworks in HE proposed by researchers over the years and the importance of tools and techniques used in TQM implementation.

Originality/value

This study, which tries to bring a perspective to the main trends in TQM application to higher education wrt models, frameworks, tools and techniques over the last thirty years, is expected to add to the body of knowledge in this area and help future researchers to focus on the relevant areas identified in this paper.

• Higher education
• Tools and techniques

Jasti, N.V.K. , Venkateswaran, V. , Kota, S. and Sangwan, K.S. (2022), "A literature review on total quality management (models, frameworks, and tools and techniques) in higher education", The TQM Journal , Vol. 34 No. 5, pp. 1298-1319. https://doi.org/10.1108/TQM-04-2021-0113

Emerald Publishing Limited

Copyright © 2021, Emerald Publishing Limited

Related articles

We’re listening — tell us what you think, something didn’t work….

Report bugs here

All feedback is valuable

Please share your general feedback

Join us on our journey

Platform update page.

Visit emeraldpublishing.com/platformupdate to discover the latest news and updates

Questions & More Information

Answers to the most commonly asked questions here

Quality of Literature Reviews

• First Online: 11 August 2022

Cite this chapter

• Rob Dekkers 4 ,
• Lindsey Carey 5 &
• Peter Langhorne 6  

1797 Accesses

Whereas the starting point of a literature review is presented in Chapter 2 —finding out more about what is written about a specific topic by evaluating it from a critical objective—, it leaves open what constitutes a good quality literature review, whether as review of scholarly knowledge before an empirical study or as stand-alone study. Keeping in mind that there are different archetypes of literature reviews, see Section  2.5 , also the way of looking at quality will vary across these types and with the objective of the literature review. Thus, it deserves a closer look at how quality of literature reviews can be assured.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as EPUB and PDF
• Read on any device
• Instant download
• Own it forever
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info
• Durable hardcover edition

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Interestingly, the publication by Webster and Watson ( 2002 ) does not dwell on the implication of the title, even though it captures the essence of a literature review.

Note that this proposition by Yin et al. ( 1976 ) is related to the formalisation of the case survey method in Yin and Heald (1975); the case survey method appears in this book as method associated with qualitative synthesis in Section  10.3 . Also note that the latter publication is a precursor to what is known now as the case study methodology.

The topical survey is also addressed by Elisabeth Bergdahl in Section  11.1 .

There is some discussion about whom introduced or modified the concepts of nomothetic and idiographic forms of generating knowledge in its early stages. On this matter, Hurlburt and Knapp ( 2006 , pp. 287–9), and Salvatore and Valsiner ( 2010 , pp. 818–20) produce slightly different accounts.

Such is found by Steenhuis & de Bruijn ( 2006 ), too, in the case which journals gravitate toward nomothetic or ideographic research.

The common term ‘craftsmanship’ has been replaced with ‘academic mastery’ to avoid any unintended connotations.

Note that Boell and Cecez-Kecmanovic ( 2010 , p. 134 ff.) introduce a search strategy that is reminiscent of the iterative search strategy, presented in Section  5.3 , rather than representative of the hermeneutic approach as detailed in the current section. In their next writing (Boell and Cecez-Kecmanovic 2014 , p. 264), the search strategy is expanded with a cycle of analysis and interpretation, closer to the analysis stages in the systematic quantitative literature review (Section  9.5 ) and content analysis (see Section  10.3 ) than to hermeneutics.

A work in point noting this confusion about the use of the term ‘offshoring’ and related wording is Jahns et al. ( 2006 , pp. 222–3); to support their interpretation, they introduce a matrix to delineate the concepts for offshoring and outsourcing.

Atkins S, Lewin S, Smith H, Engel M, Fretheim A, Volmink J (2008) Conducting a meta-ethnography of qualitative literature: lessons learnt. BMC Med Res Methodol 8(1):21. https://doi.org/10.1186/1471-2288-8-21

Article   Google Scholar  

Avellar SA, Thomas J, Kleinman R, Sama-Miller E, Woodruff SE, Coughlin R, Westbrook TPR (2017) External validity: the next step for systematic reviews? Eval Rev 41(4):283–325. https://doi.org/10.1177/0193841x16665199

Babakus WS, Thompson JL (2012) Physical activity among South Asian women: a systematic, mixed-methods review. Int J Behav Nutr Phys Act 9(1):150. https://doi.org/10.1186/1479-5868-9-150

Bailey R, Pearce G, Smith C, Sutherland M, Stack N, Winstanley C, Dickenson M (2012) Improving the educational achievement of gifted and talented students: a systematic review. Talent Dev Excel 4(1):33–48

Google Scholar  

Bearman M (2016) Quality and literature reviews: beyond reporting standards. Med Educ 50(4):382–384. https://doi.org/10.1111/medu.12984

Bengtsson L, Elg U, Lind J-I (1997) Bridging the transatlantic publishing gap: how North American reviewers evaluate European idiographic research. Scand J Manag 13(4):473–492. https://doi.org/10.1016/S0956-5221(97)00022-5

Bergdahl E (2019) Is meta-synthesis turning rich descriptions into thin reductions? A criticism of meta-aggregation as a form of qualitative synthesis. Nurs Inq 26(1):e12273. https://doi.org/10.1111/nin.12273

Berkovich I (2018) Beyond qualitative/quantitative structuralism: the positivist qualitative research and the paradigmatic disclaimer. Qual Quant 52(5):2063–2077. https://doi.org/10.1007/s11135-017-0607-3

Bodolica V, Spraggon M (2018) An end-to-end process of writing and publishing influential literature review articles: do’s and don’ts. Manag Decis 56(11):2472–2486. https://doi.org/10.1108/MD-03-2018-0253

Boell SK, Cecez-Kecmanovic D (2010) Literature reviews and the hermeneutic circle. Aust Acad Res Libr 41(2):129–144. https://doi.org/10.1080/00048623.2010.10721450

Boell SK, Cecez-Kecmanovic D (2014) A hermeneutic approach for conducting literature reviews and literature searches. Commun Assoc Inf Syst 34:257–286. https://doi.org/10.17705/1CAIS.03412

Boell SK, Cecez-Kecmanovic D (2015) On being ‘systematic’ in literature reviews in IS. J Inf Technol 30(2):161–173. https://doi.org/10.1057/jit.2014.26

Bolderston A (2008) Writing an effective literature review. J Med Imaging Radiat Sci 39(2):86–92. https://doi.org/10.1016/j.jmir.2008.04.009

Borras SM, Hall R, Scoones I, White B, Wolford W (2011) Towards a better understanding of global land grabbing: an editorial introduction. J Peasant Stud 38(2):209–216. https://doi.org/10.1080/03066150.2011.559005

Brandstätter M, Baumann U, Borasio GD, Fegg MJ (2012) Systematic review of meaning in life assessment instruments. Psychooncology 21(10):1034–1052. https://doi.org/10.1002/pon.2113

Brereton P, Kitchenham BA, Budgen D, Turner M, Khalil M (2007) Lessons from applying the systematic literature review process within the software engineering domain. J Syst Softw 80(4):571–583. https://doi.org/10.1016/j.jss.2006.07.009

Campbell R, Pound P, Pope C, Britten N, Pill R, Morgan M, Donovan J (2003) Evaluating meta-ethnography: a synthesis of qualitative research on lay experiences of diabetes and diabetes care. Soc Sci Med 56(4):671–684. https://doi.org/10.1016/S0277-9536(02)00064-3

de Groot AD (1969) Methodology: foundations of inference and research in the behavioral sciences. The Hague, Mouton

de Moura DA, Botter RC (2017) Toyota production system—one example to shipbuilding industry. Indep J Manag Prod 8(3):874–897. https://doi.org/10.14807/ijmp.v8i3.626

Dekkers R (2005) (R)Evolution, organizations and the dynamics of the environment. Springer, New York

Dekkers R (2017) Applied systems theory, 2nd edn. Springer, Cham

Dekkers R, Kühnle H (2012) Appraising interdisciplinary contributions to theory for collaborative (manufacturing) networks: still a long way to go? J Manuf Technol Manag 23(8):1090–1128. https://doi.org/10.1108/17410381211276899

Delllinger AB (2005) Validity and the review of literature. Res Sch 12(2):41–54

Denzin NK, Lincoln YS (1994) Handbook of qualitative research. Sage, Thousands Oaks, CA

Dixon-Woods M, Agarwal S, Jones D, Young B, Sutton A (2005) Synthesising qualitative and quantitative evidence: a review of possible methods. J Health Serv Res Policy 10(1):45–53b. https://doi.org/10.1258/1355819052801804

Dwan K, Altman DG, Arnaiz JA, Bloom J, Chan A-W, Cronin E, Williamson PR (2008) Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS One 3(8):e3081. https://doi.org/10.1371/journal.pone.0003081

Ellaway RH, O’Gorman L, Strasser R, Marsh DC, Graves L, Fink P, Cervin C (2016) A critical hybrid realist-outcomes systematic review of relationships between medical education programmes and communities: BEME Guide No. 35. Med Teach 38(3):229–245. https://doi.org/10.3109/0142159X.2015.1112894

Estabrooks CA, Field PA, Morse JM (1994) Aggregating qualitative findings: an approach to theory development. Qual Health Res 4(4):503–511. https://doi.org/10.1177/104973239400400410

Furunes T (2019) Reflections on systematic reviews: moving golden standards? Scand J Hosp Tour 19(3):227–231. https://doi.org/10.1080/15022250.2019.1584965

Gadamer H-G (1975) The problem of historical consciousness. Grad Fac Philos J 5(1):8–52. https://doi.org/10.5840/gfpj1975512

Galati G, Moessner R (2013) Macroprudential policy—a literature review. J Econ Surv 27(5):846–878. https://doi.org/10.1111/j.1467-6419.2012.00729.x

Gomez-Mejia LR, Cruz C, Berrone P, De Castro J (2011) The bind that ties: socioemotional wealth preservation in family firms. Acad Manag Ann 5(1):653–707. https://doi.org/10.5465/19416520.2011.593320

Gough D, Elbourne D (2002) Systematic research synthesis to inform policy, practice and democratic debate. Soc Policy Soc 1(3):225–236. https://doi.org/10.1017/S147474640200307X

Granello DH (2001) Promoting cognitive complexity in graduate written work: using Bloom’s taxonomy as a pedagogical tool to improve literature reviews. Couns Educ Superv 40(4):292–307. https://doi.org/10.1002/j.1556-6978.2001.tb01261.x

Grondin J (2015) The hermeneutical circle. In: Keane N, Lawn C (eds) The Blackwell companion to hermeneutics. Wiley, Chichester, pp 299–305

Chapter   Google Scholar  

Guba EG, Lincoln YS (1994) Competing paradigms in qualitative research. In: Denzin NK, Lincoln YS (eds) Handbook of qualitative research. Sage, Thousand Oaks, CA, pp 105–117

Hagedoorn J, Duysters G (2002) External sources of innovative capabilities: the preferences for strategic alliances or mergers and acquisitions. J Manag Stud 39(2):167–188

Hurlburt RT, Knapp TJ (2006) Münsterberg in 1898, not Allport in 1937, introduced the terms ‘idiographic’ and ‘nomothetic’ to American psychology. Theory Psychol 16(2):287–293. https://doi.org/10.1177/0959354306062541

Jahns C, Hartmann E, Bals L (2006) Offshoring: dimensions and diffusion of a new business concept. J Purch Supply Manag 12(4):218–231. https://doi.org/10.1016/j.pursup.2006.10.001

Janesick VJ (1994) The dance of qualitative research design: metaphor, methodolatry, and meaning. In: Denzin NK, Lincoln YS (eds) Handbook of qualitative research. Sage, Thousand Oaks, CA, pp 209–219

Jungherr A (2016) Twitter use in election campaigns: a systematic literature review. J Inform Tech Polit 13(1):72–91. https://doi.org/10.1080/19331681.2015.1132401

Karakas F (2010) Spirituality and performance in organizations: a literature review. J Bus Ethics 94(1):89–106. https://doi.org/10.1007/s10551-009-0251-5

Kennedy MM (2007) Defining a literature. Educ Res 36(3):139–147. https://doi.org/10.3102/0013189x07299197

Kvale S (1995) The social construction of validity. Qual Inq 1(1):19–40. https://doi.org/10.1177/107780049500100103

Lawrence M, Kerr S, McVey C, Godwin J (2012) The effectiveness of secondary prevention lifestyle interventions designed to change lifestyle behavior following stroke: summary of a systematic review. Int J Stroke 7(3):243–247. https://doi.org/10.1111/j.1747-4949.2012.00771.x

Lin AC (1998) Bridging positivist and interpretivist approaches to qualitative methods. Policy Stud J 26(1):162–180. https://doi.org/10.1111/j.1541-0072.1998.tb01931.x

Long HA, French DP, Brooks JM (2020) Optimising the value of the critical appraisal skills programme (CASP) tool for quality appraisal in qualitative evidence synthesis. Res Methods Med Health Sci 1(1):31–42. https://doi.org/10.1177/2632084320947559

Maier HR (2013) What constitutes a good literature review and why does its quality matter? Environ Model Softw 43:3–4. https://doi.org/10.1016/j.envsoft.2013.02.004

Mays N, Pope C (2000) Assessing quality in qualitative research. BMJ 320(7226):50–52. https://doi.org/10.1136/bmj.320.7226.50

McKercher B, Law R, Weber K, Song H, Hsu C (2007) Why referees reject manuscripts. J Hosp Tour Res 31(4):455–470. https://doi.org/10.1177/1096348007302355

Mohammed MA, Moles RJ, Chen TF (2016) Meta-synthesis of qualitative research: the challenges and opportunities. Int J Clin Pharm 38(3):695–704. https://doi.org/10.1007/s11096-016-0289-2

Montuori A (2005) Literature review as creative inquiry: reframing scholarship as a creative process. J Transform Educ 3(4):374–393. https://doi.org/10.1177/1541344605279381

Moja LP, Telaro E, D’Amico R, Moschetti I, Coe L, Liberati A (2005) Assessment of methodological quality of primary studies by systematic reviews: results of the metaquality cross sectional study. BMJ 330(7499):1053. https://doi.org/10.1136/bmj.38414.515938.8F

Münsterberg H (1899) Psychology and history. Psychol Rev VI(I):1–31. https://doi.org/10.1037/h0071306

Munthe-Kaas H, Nøkleby H, Lewin S, Glenton C (2020) The TRANSFER approach for assessing the transferability of systematic review findings. BMC Med Res Methodol 20(1):11. https://doi.org/10.1186/s12874-019-0834-5

Munthe-Kaas H, Nøkleby H, Nguyen L (2019) Systematic mapping of checklists for assessing transferability. Syst Rev 8(1):22. https://doi.org/10.1186/s13643-018-0893-4

Nakano D, Muniz Jr, J (2018) Writing the literature review for empirical papers. Production 28:e20170086. https://doi.org/10.1590/0103-6513.20170086

Oakes G (1980) History and Natural Science. Hist Theory 19(2):165–168. https://doi.org/10.2307/2504797

Oxman AD, Guyatt GH (1988) Guidelines for reading literature reviews. Can Med Assoc J 138(8):697–703

Petty R, Guthrie J (2000) Intellectual capital literature review: measurement, reporting and management. J Intellect Cap 1(2):155–176. https://doi.org/10.1108/14691930010348731

Pluye P, Gagnon M-P, Griffiths F, Johnson-Lafleur J (2009) A scoring system for appraising mixed methods research, and concomitantly appraising qualitative, quantitative and mixed methods primary studies in mixed studies reviews. Int J Nurs Stud 46(4):529–546. https://doi.org/10.1016/j.ijnurstu.2009.01.009

Popper K (1999) All life is problem solving. Routledge, London

Ribeiro ÍJS, Pereira R, Freire IV, de Oliveira BG, Casotti CA, Boery EN (2018) Stress and quality of life among university students: a systematic literature review. Health Prof Educ 4(2):70–77. https://doi.org/10.1016/j.hpe.2017.03.002

Robinson OC (2011) The idiographic/nomothetic dichotomy: tracing historical origins of contemporary confusions. Hist Philos Psychol 13(2):32–39

Rowley J, Slack F (2004) Conducting a literature review. Manag Res News 27(6):31–39. https://doi.org/10.1108/01409170410784185

Salvatore S, Valsiner J (2010) Between the general and the unique: overcoming the nomothetic versus idiographic opposition. Theory Psychol 20(6):817–833. https://doi.org/10.1177/0959354310381156

Schwandt TA (1994) Constructivist, interpretivist approaches to human inquiry. In: Denzin NK, Lincoln YS (eds) Handbook of qualitative research. Sage, Thousand Oaks, CA, pp 118–137

Shephard K, Rieckmann M, Barth M (2019) Seeking sustainability competence and capability in the ESD and HESD literature: an international philosophical hermeneutic analysis. Environ Educ Res 25(4):532–547. https://doi.org/10.1080/13504622.2018.1490947

Smythe E, Spence D (2012) Re-viewing literature in hermeneutic research. Int J Qual Methods 11(1):12–25. https://doi.org/10.1177/160940691201100102

Snyder H (2019) Literature review as a research methodology: an overview and guidelines. J Bus Res 104:333–339. https://doi.org/10.1016/j.jbusres.2019.07.039

Spencer L, Ritchie J, Lewis J, Dillon L (2003) Quality in qualitative evaluation: a framework for assessing research evidence. Cabinet Office, London

Steenhuis HJ, de Bruijn EJ (2006) Publishing in OM: does scientific paradigm matter? In Annual Meeting of the Academy of Management, Atlanta, CA, 11–16 August 2016

Strang KD (2015) Articulating a research design ideology. In: Strang KD (ed) The Palgrave handbook of research design in business and management. Palgrave Macmillan, New York, NY, pp 17–30

Tilden T (2020) The idiographic voice in a nomothetic world: why client feedback is essential in our professional knowledge. In: Ochs M, Borcsa M, Schweitzer J (eds) Systemic research in individual, couple, and family therapy and counseling. Springer International Publishing, Cham, pp 385–399

van Laar E, van Deursen AJAM, van Dijk JAGM, de Haan J (2017) The relation between 21st-century skills and digital skills: a systematic literature review. Comput Hum Behav 72:577–588. https://doi.org/10.1016/j.chb.2017.03.010

Wagenmakers E-J, Dutilh G, Sarafoglou A (2018) The creativity-verification cycle in psychological science: new methods to combat old idols. Perspect Psychol Sci 13(4):418–427. https://doi.org/10.1177/1745691618771357

Walsh D, Downe S (2006) Appraising the quality of qualitative research. Midwifery 22(2):108–119. https://doi.org/10.1016/j.midw.2005.05.004

Webster J, Watson RT (2002) Analyzing the past to prepare for the future: writing a literature review. MIS Q 26(2):xiii–xxiii

Yin RK, Bingham E, Heald KA (1976) The difference that quality makes: the case of literature reviews. Sociol Methods Res 5(2):139–156. https://doi.org/10.1177/004912417600500201

Yin RK, Heald KA (1975) Using the case survey method to analyze policy studies. Adm Sci Q 20(3):371–381. https://doi.org/10.2307/2391997

Zhou J, Li X, Mitri HS (2018) Evaluation method of rockburst: state-of-the-art literature review. Tunn Undergr Space Technol 81:632–659. https://doi.org/10.1016/j.tust.2018.08.029

Download references

Author information

Authors and affiliations.

University of Glasgow, Glasgow, UK

Rob Dekkers

Glasgow Caledonian University, Glasgow, UK

Lindsey Carey

Prof. Peter Langhorne

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Rob Dekkers .

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this chapter

Dekkers, R., Carey, L., Langhorne, P. (2022). Quality of Literature Reviews. In: Making Literature Reviews Work: A Multidisciplinary Guide to Systematic Approaches. Springer, Cham. https://doi.org/10.1007/978-3-030-90025-0_3

Download citation

DOI : https://doi.org/10.1007/978-3-030-90025-0_3

Published : 11 August 2022

Publisher Name : Springer, Cham

Print ISBN : 978-3-030-90024-3

Online ISBN : 978-3-030-90025-0

eBook Packages : Education Education (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Login | Register

• Editorial Team

An In-Depth Literature Review of E-Portfolio Implementation in Higher Education: Processes, Barriers, and Strategies

Authors: Hongyan Yang (The University of Tennessee, Knoxville) , Rachel Wong (The University of Tennessee, Knoxville)

Literature Review

• Harvard Citation Style
• Vancouver Citation Style
• APA Citation Style
• Download RIS
• Download BibTeX

This is an accepted article with a DOI pre-assigned that is not yet published.

This literature review examines the implementation of e-portfolios in higher education, with a focus on the implementation process, potential barriers, and strategies for overcoming challenges. This review seeks to provide instructional designers and higher education instructors with design strategies to effectively implement e-portfolios. Through an analysis of seventeen studies, we identified six common steps in the implementation process, including identifying a purpose, stakeholders, and platform, conducting workshops, creating e-portfolios, and evaluating the project. The implementation process also raised eight concerns, including concerns related to technology, policy, pedagogy, artifact quality, privacy, student motivation, academic integrity, and teacher workload. To address these concerns, existing strategies suggest that successful implementation requires training and policy support, student-centered pedagogy, criteria for assessing artifacts, privacy and data protection, feedback, anti-plagiarism measures, and shared successful models.

Keywords: literature review, e-Portfolio, implementation, higher education

Accepted on 20 Apr 2024

Peer reviewed, creative commons attribution-noncommercial-sharealike 4.0, harvard-style citation.

Yang, H & Wong, R. () 'An In-Depth Literature Review of E-Portfolio Implementation in Higher Education: Processes, Barriers, and Strategies', Issues and Trends in Learning Technologies . doi: 10.2458/itlt.5809

Show: Vancouver Citation Style | APA Citation Style

Vancouver-Style Citation

Yang, H & Wong, R. An In-Depth Literature Review of E-Portfolio Implementation in Higher Education: Processes, Barriers, and Strategies. Issues and Trends in Learning Technologies. ; doi: 10.2458/itlt.5809

Show: Harvard Citation Style | APA Citation Style

APA-Style Citation

Yang, H & Wong, R. (, ). An In-Depth Literature Review of E-Portfolio Implementation in Higher Education: Processes, Barriers, and Strategies. Issues and Trends in Learning Technologies doi: 10.2458/itlt.5809

Show: Harvard Citation Style | {% trans 'Vancouver Citation Style' %}

Non Specialist Summary

This article has no summary