research paper 3d

A Survey of 3D Printing Technologies as Applied to Printed Electronics

Ieee account.

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Materials (Basel)

An Introduction to 3D Bioprinting: Possibilities, Challenges and Future Aspects

Željka p. kačarević.

1 Department of Anatomy Histology, Embryology, Pathology Anatomy and Pathology Histology, Faculty of Dental Medicine and Health, University of Osijek, 31000 Osijek, Croatia

Patrick M. Rider

2 Botiss Biomaterials, Hauptstraße 28, 15806 Zossen, Germany; [email protected] (P.M.R.); [email protected] (M.B.)

Said Alkildani

3 Department of Biomedical Engineering, Faculty of Applied Medical Sciences, German-Jordanian University, 11180 Amman, Jordan; moc.liamg@inadlikdias

Sujith Retnasingh

4 Institute for Environmental Toxicology, Martin-Luther-Universität, Halle-Wittenberg and Faculty of Biomedical Engineering, Anhalt University of Applied Science, 06366 Köthen, Germany; moc.liamg@ihsorijus

Ralf Smeets

5 Department of Oral and Maxillofacial Surgery, University Hospital Hamburg-Eppendorf, 20246 Hamburg, Germany; [email protected]

6 Department of Oral Maxillofacial Surgery, Division of Regenerative Orofacial Medicine, University Medical Center Hamburg-Eppendorf, 20246 Hamburg, Germany; [email protected]

Zrinka Ivanišević

7 Department of Dental Medicine, Faculty of Dental Medicine and Health, University of Osijek, 31000 Osijek, Croatia; moc.liamg@naviaknirz

Mike Barbeck

8 BerlinAnalytix GmbH, 12109 Berlin, Germany

Bioprinting is an emerging field in regenerative medicine. Producing cell-laden, three-dimensional structures to mimic bodily tissues has an important role not only in tissue engineering, but also in drug delivery and cancer studies. Bioprinting can provide patient-specific spatial geometry, controlled microstructures and the positioning of different cell types for the fabrication of tissue engineering scaffolds. In this brief review, the different fabrication techniques: laser-based, extrusion-based and inkjet-based bioprinting, are defined, elaborated and compared. Advantages and challenges of each technique are addressed as well as the current research status of each technique towards various tissue types. Nozzle-based techniques, like inkjet and extrusion printing, and laser-based techniques, like stereolithography and laser-assisted bioprinting, are all capable of producing successful bioprinted scaffolds. These four techniques were found to have diverse effects on cell viability, resolution and print fidelity. Additionally, the choice of materials and their concentrations were also found to impact the printing characteristics. Each technique has demonstrated individual advantages and disadvantages with more recent research conduct involving multiple techniques to combine the advantages of each technique.

1. Introduction

Bioprinting is a subcategory of additive manufacturing (AM), also known as three-dimensional (3D) printing. It is defined as the printing of structures using viable cells, biomaterials and biological molecules [ 1 , 2 ]. Bioprinting must produce scaffolds with a suitable microarchitecture to provide mechanical stability and promote cell ingrowth whilst also considering the impact of manufacture on cell viability; for instance, chemical cytotoxicity caused by the use of solvents or pressure-induced apoptotic effect produced during the extrusion of material. A significant benefit of bioprinting is that it prevents homogeneity issues that accompany post-fabrication cell seeding, as cell placement is included during fabrication.

The advantage of homogeneously distributed cell-laden scaffolds has been demonstrated by faster integration with the host tissue, lower risk of rejection and most importantly, uniform tissue growth in vivo [ 3 , 4 , 5 , 6 ]. Conventional cell seeding techniques are either static or dynamic, and while the latter one results in better seeding efficiency and cell penetration into the scaffold, it is known affect cell morphology [ 7 ].

Immediate vascularization of the implanted scaffolds is highly critical [ 8 , 9 ]. With proper vascularization, the scaffolds are provided with an influx of oxygen/nutrients and an efflux of carbon dioxide/by-products; preventing core necrosis. Vascularization also supports the implants with remodelling [ 10 ]. Bioprinting techniques have been employed to fabricate microvascular-like structures and have the potential to position endothelial cells within the 3D structures as a prevascularization step prior to implantation [ 11 ].

Bioprinting can be applied in a clinical setting, where it can be used to create regenerative scaffolds to suit patient specific requirements [ 12 ]. The process of applying bioprinting to a clinical setting is depicted in Figure 1 . To begin with, imaging modalities such as CT, MRI and ultrasound can be used to create a digital 3D model of the tissue defect. Using computer aided design (CAD), the internal and external architecture of the scaffold, such as porosity and pore sizes, can be incorporated into the 3D model of the tissue defect. In consideration of the defect type, location and requirements, a selection of materials, cell types and bioactive molecules, can be used to fabricate a bioink for printing. Cell laden structures are then manufactured using bioprinting technology and are then placed either in cell culture or directly implanted into the patient.

Schematic of Bioprinting Scaffolds for clinical use. Digital 3D images obtained from CT, MRI or ultrasound, are used to design a suitable scaffold with 3D slicing and CAD software; materials from printing are chosen depending upon the application, and can consist of polymers, ceramics, and bioactive components; cells are selected dependent on the application, a bioink can consist of singular or multiple cell types; post-fabrication 3D culture can be used for characterization, assessment and ultimately implantation. 3D printing is both time and cost effective, enabling fast adjustments and implementation of designs [ 13 ]. Designs can be made to match exact defect geometries, improving the union between implant and native tissue, thereby enhancing tissue integration [ 14 ]. Additive manufactured scaffolds have shown satisfactory accuracy matching the designs [ 15 , 16 , 17 ]. Different types of tissues and organs have been produced using bioprinting, for instance; blood vessels [ 18 ], heart tissue [ 19 ], skin [ 20 , 21 ], liver tissue [ 5 ], neural tissue [ 22 ], cartilage [ 23 ] and bone [ 24 ].

The ultimate aim of bioprinting is to provide an alternative to autologous and allogeneic tissue implants, as well as to replace animal testing for the study of disease and development of treatments. In this review, the main bioprinting techniques are discussed: inkjet-based, extrusion-based and laser-assisted, including their basic mechanisms and current challenges. Table 1 , Table 2 and Table 3 provide an overview of recent research for each technique.

Recent in vitro studies. AG—Agarose, SA—Sodium alginate, PLA—Polylactide fibers, GelMA—gelatin methacryloyl, HUVECs—Human umbilical vein endothelial cells, PEGDA—poly(ethylene glycol) diacrylate, ATCC—Mouse neural stem cell lines, BrCa—breast cancer cells, MSCs—marrow mesenchymal stem cells, Nha—nanocrystalline hydroxyapatite.

Recent in vivo studies. Abbreviations: PU—poly(urethane), PCL—poly(caprolactone), hASCs—human adipose-derived stem cells, NSCs—neural stem cells, PEG—poly(ethylene glycol), HUVECs—human umbilical vein endothelial cells, iPSCs—induced pluripotent stem cells, CM—cardiomyocytes, bMSCs—bone marrow-derived mesenchymal stem cells, ROB—rat osteoblasts, TCP—tricalcium phosphates, HMECs—human microvascular endothelial cells.

Recent in situ studies. Abbreviations: IPFP—Human infrapatellar fat pad-derived adipose stem cells, GelMA—gelatin methacryloyl, HAMa—hyaluronic acid–methacrylate hydrogel, PEGDMA—Poly(ethylene glycol) dimethacrylate, AFS—Amniotic fluid-derived stem cells, MSCs—bone marrow-derived mesenchymal stem cells.

An important component of bioprinting is the use of bioinks. Bioinks consist of biomaterials that can be used to encapsulate cells and incorporate biomolecules. Cell laden bioinks are hydrogel-based, as hydrogels have a high water content that is beneficial for cell survivability and shielding the cells from fabrication induced forces. The main properties of a bioink that need to be considered before printing include its viscosity, gelation and crosslinking capabilities. These properties can significantly affect print fidelity (construct stability and print deviation from the computer aided designs) as well as cell viability, proliferation and morphology after printing [ 25 ]. To produce a hydrogel that can both support and protect the cells, whilst at the same time provide a structurally secure scaffold is challenging, as these characteristics have different mechanical requirements. Stiff hydrogels have denser networks that might put the cells under pressure during encapsulation, as well as hinder their migration [ 26 ]. Ultimately, the hydrogel properties need to be balanced between structural fidelity and cell suspension.

2. Inkjet-Based Bioprinting

First attempts to print live cells was performed using a specially adapted commercially available inkjet printers [ 1 ]. An initial problem encountered when developing inkjet bioprinting was that the cells died during printing due to instantaneous drying out once on the substrate. The problem was overcome by encapsulating the cells in a highly hydrated polymer, this led to the development of cell-loaded hydrogels [ 48 ]. Inkjet bioprinting allows for the precise positioning of cells, with some studies achieving as few as a singular cell per printed droplet [ 49 ]. Cells and biomaterials are patterned into a desired pattern using droplets, ejected via thermal or piezoelectric processes, depicted in Figure 2 [ 1 , 50 ].

Schematic of Inkjet-based Bioprinting. Thermal inkjet uses heat-induced bubble nucleation that propels the bioink through the micro-nozzle. Piezoelectric actuator produces acoustic waves that propel the bioink through the micro-nozzle.

Thermal-based inkjet printing uses a heated element to nucleate a bubble. The bubble causes a build-up pressure within the printhead, which leads to the expulsion of a droplet. The thermal element can reach temperatures between 100 °C to 300 °C. Initially there have been concerns that such high temperatures would damage the cells [ 51 ], however research has shown that the high temperatures are localized and are only present for a short time span [ 11 , 52 ].

Piezoelectric-based apparatus uses acoustic waves to eject the bioink. This mechanism limits the use of highly concentrated and viscous bioinks as their viscosity dampens the applied acoustic/pressure waves, hindering the ejection of a droplet [ 53 ]. A low viscosity is achieved by using low concentration solutions, a limiting factor for producing 3D structures [ 50 ].

Inkjet printing offers a high resolution of up to 50 µm [ 54 ]. Most inkjet bioprinters provide a high cell viability, and although there is the potential for induced sheer stresses to damage the cells, most research indicates that this is not the case [ 55 , 56 ]. The advantages of inkjet-based bioprinting include high print speeds, low cost and a wide availability, however problems include low droplet directionality and unreliable cell encapsulation due to the low concentration of the ink [ 1 ].

Cui et al. developed a 3D printed bone-like tissue using poly (ethylene glycol) dimethacrylate (PEGDMA), that had a similar compressive modulus to natural bone, and bioceramic nanoparticles [ 57 ]. Human mesenchymal stem cells (hMSCs), PEGDMA with hydroxyapatite (HA) and/or bioglass (BG) nanoparticles were bioprinted into bone tissue scaffolds. The bioceramic nanoparticles were used to mimic the native bone tissue microenvironment and stimulated the differentiation of stem cells towards osteogenic linage. There was significant difference between compressive mechanical strengths of pure PEG and PEG-HA scaffolds (~0.35 MPa); however, mechanical strength dropped significantly for PEG-BG scaffolds. Incubation of scaffolds in cell culture for 21 days seemed to increase modulus in all samples except for PEG-BG. The interaction of hMSCs and HA nanoparticles produced highest cell viability of 86% compared to the other scaffolds.

Inkjet bioprinting has demonstrated excellent cell viabilities and the potential for creating a neural network in printed organs. Tse et al. fabricated neural tissue by bioprinting porcine Schwann cells and neuronal NG 108-15 cells using a piezoelectric inkjet printer [ 32 ]. Neuronal and glial cell viabilities of 86% and 90% were observed immediately after printing. Proliferation rate of the printed cells was close to those which weren’t printed. The printed cells seemed to have developed neurites that elongated after 7 days.

Cardiac tissue with a beating cell response was engineered by Aho et al. using feline cardiomyocytes HI.1 cardiac muscle cells and an alginate hydrogel. The tissue was fabricated by printing layers of CaCl 2 into an alginate hydrogel precursor solution to facilitate crosslinking. The results suggested that cardiac cells attached to the alginate, effectively mimicked the native cardiac ECM. The printed cardiac tissues exhibited contractile properties under mild electrical stimuli [ 33 ].

Min et al. fabricated full thickness skin models with pigmentation using an inkjet technique [ 58 ]. Dermal models was fabricated from fibroblast-laden collagen. After culturing for 1 day in fibroblast medium, keratinocytes were printed on top of the dermal model and put in culture for another day. Melanocytes were then printed onto the model and further cultured in melanocyte medium for 2 h. The entire model was subjected to air-liquid-interface for 4 days. The construct had distinctive epidermal and dermal layers. Keratinocytes reached maturation and melanocytes resulted in freckle-like pigmentation (without chemical or UV stimuli). Sodium carbonate was used for crosslinking.Yanez et al. investigated the wound healing capabilities of bioprinted skin grafts [ 59 ]. Skin grafts were fabricated by printing fibrinogen solution onto to a layer of collagen that was laden with human dermal fibroblasts (NHDFs). A subsequent layer of thrombin, laden with human dermal microvascular endothelial cells (HMVECs) was bioprinted onto the fibrinogen. Finally, collagen laden with neonatal human epidermal keratinocytes (NHEKs) was printed onto the fibrin-HMVEC layer. The grafts were incubated for 24 h and transplanted subcutaneously in to the backs of mice. Wounds treated with the bioprinted scaffold had completely healed after 14–16 days, whereas wounds treated without the graft healed in 21 days.

Inkjet bioprinting is of great interest as it exhibits high resolution and cell viability. With this process, accurate position of multiple cell types is possible [ 49 , 60 ]. However, the limitations of vertical printing and restricted viscosities may mean that inkjet bioprinting needs to be combined with other printing techniques for future developments.

3. Laser-Based Bioprinting

Stereolithography (SLA) is an AM technique that uses ultraviolet (UV) or visible light to cure photosensitive polymers in a layer-by-layer fashion, as shown in Figure 3 . This nozzle-free technique eliminates the negative effects of shear pressure encountered when using nozzle-based bioprinting. It offers a fast and accurate fabrication, with resolutions ranging between 5–300 µm [ 61 , 62 ]. Polymerization occurs at the top of the bioink vat where the biomaterial is exposed to the light energy. After each layer is polymerized, the platform supporting the structure will be lowered in the vat, enabling a new layer to be photopolymerized on top.

Schematic of Stereolithography Bioprinting. Photopolymerization occurs on the surface of the vat where the light-sensitive bioink is exposed to light energy. Axial platform moves downward the Z-axis during fabrication. This layer-by-layer technique does not depend on the complexity of the design, rather on its height.

Photoinitiators are chemical molecules that create reactive agents when exposed to light energy, which react with monomers of a material to then initiate the formation of polymer chains. Photoinitiators are sensitive to different ranges of wavelength; some are triggered by UV and others by visible light. The stiffness and network density of the cured resin depends on the concentration of the photointiator but higher concentrations might exhibit adverse cytotoxic effects. However, different photoinitiators have different cytotoxicity levels. The most commonly used and the least cytotoxic photoinitiators are Irgacure 2959 for UV cross-linkage and eosin Y for visible light [ 63 ]. Eosin Y has even shown to be less toxic than Irgacure 2959 [ 63 ]. UV light will affect cells and introduce mutations [ 64 ]; therefore, visible light-based photocross-linkage has been adopted more frequently in SLA as well as in situ applications [ 65 , 66 ]. Photopolymerization is also employed during or post-fabrication via inkjet- and extrusion-based printing to harden the prints [ 26 , 57 ].

Due to the risk of damaging the cells through the use of UV light or cytotoxic effects of the photoinitiators, several researchers have investigated alternative means to enable photopolymerization of bioinks. Hoffmann et al. developed a class of materials that crosslink without the presence of a photoinitiator using a thiol-ene reaction [ 67 ]. The used monomers comprise two classes of monomers containing at least two alkene or thiol groups. These two components react spontaneously under ultraviolet (UV)-irradiation at a wavelength of approximately 266 nm. A 1:1 ratio of thiol and alkene exhibited high cell viability after 3 days, ≈95%. However, doubling the thiol content resulted in a cytotoxic effect, even though this amount of thiol groups provides high amounts of surface functional groups, allowing greater subsequent surface functionalization.

Zhang et al. used UV laser in the form of Bessel beam [ 68 ]. Bessel beam does not diffract and spread out, which will be useful to increase print fidelity and decrease fabrication time. The precursor hydrogel was prepared from GelMA, PEGMA and Irgacure 2959. Human umbilical vein endothelial cells (HUVECs) were encapsulated in the hydrogel. Cell-laden fibers with diameters 25, 43 and 75 µm were fabricated and cell viability was 95% after 3 days. This technique has potential in fabricating tubular constructs and porous scaffolds under a shortened fabrication time; however, is limited to low structural complexity.

Tuan et al. developed a visible light-based stereolithography using Lithium phenyl-2,4,6-trimethylbenzoylphosphinate (LAP) [ 69 ], which is a UV-sensitive photoinitiator that can also respond to near-UV blue light [ 63 ]. Human adipose-derived stem cells (hADSCs) were suspended in a Poly(ethylene glycol) diacrylate (PEGDA)/LAP solution. Although near-UV blue light, 400–490 nm can be damaging to mammalian cells [ 70 ], after fabrication the hADSCs exhibited a high metabolic activity, increasing by 75% and 50% after 5 and 7 days, respectively.

Other photoinitiators that can absorb visible light are camphorquinone and eosin Y, that crosslink at wavelengths of 400–700 nm and 514 nm, respectively [ 63 ]. Wang et al. mixed PEG with eosin Y and methacrylated gelatin (GelMA). Samples without GelMA exhibited decreased cell viability compared to the samples consisting of 5% and 7.5% GelMA, which maintained cell viabilities of ~80% after 5 days [ 71 ]. The slightly decreased cell viability could be related to the fact that PEG is non-adhesive, causing the death of anchorage-dependent cells [ 72 , 73 ].

Wang et al. fabricated GelMA-based scaffolds via visible light-based SLA [ 74 ]. The precursor gel was mixed with eosin Y and NIH-3T3 fibroblasts. The scaffolds were crosslinked by a commercial projector at 522 nm wavelength. After 5 days in culture, most of the cells adhered to bioink.

Hu et al. studied the cytotoxicity of chitosan-based scaffolds that were mixed with either camphorquinone, fluorescein or riboflavin [ 75 ]. Fluorescein and riboflavin are blue light-absorbing initiators. Camphorquinone exhibited relatively low cell viability, ~40%, whilst the other two photoinitiators exhibited cell viabilities >80%. Camphorquinone is more commonly used than the other two photoinitiators; however, biocompatibility results of camphorquinone have been inconsistent in literature [ 75 , 76 , 77 ].

Stereolithography has much to offer in its application to bioprinting. The absence of shear stress and no limitation on bioink viscosity make it as an appealing choice for incorporating cells within scaffolds. However, the limitations of SLA include the damage caused by UV and near UV light to cell DNA, the limited choice of photosensitive biomaterials as well as the cytotoxicity of added. Some researchers have already begun to look for alternatives, such as using photoinitiator-free materials or visible light-absorbing photoinitiators [ 67 , 78 ].

4. Laser-Assisted Bioprinting

Laser-assisted printing was initially developed to deposit metals onto receiver sheets [ 79 , 80 ]. Odde and Renn later developed the technique to print viable embryonic chick spinal cord cells [ 81 ]. Laser-assisted bioprinting (LAB) consists of three parts: a donor-slide (or ribbon), a laser pulse and a receiver-slide. A ribbon is made of a layer of transparent glass, a thin layer of metal, and a layer of bioink. The bioink is transferred from the ribbon onto the receiver slide when the metal layer under the hydrogel is vaporized by a laser pulse, as depicted in Figure 4 . This scaffold-free technique has very high cell viabilities (>95 [ 54 ]) and a resolution between 10–50 µm [ 1 ]. Some studies using LAB have demonstrated an accuracy of a singular cell per droplet [ 82 ].

Schematic of Laser-assisted Bioprinting. ( a ) transparent glass, ( b ) thin metal layer, ( c ) vaporization-induced bubble. Bubble nucleation induced by laser energy propels droplets of bioink towards the substrate. This technique has minimal effect on cell viability. A receiver-slide can be a biopaper, polymer sheet or scaffold.

Gruene et al. conducted a study to observe the effects of the LAB laser pulse had on printed mesenchymal stems cells (MSCs). It was found that the laser pulse had a negligible effect. There were no reported changes in gene expression caused by the heat shock of the laser pulse, and cell proliferation rates were as high as the control of non-printed cells after 5 days in cell culture [ 24 ]. Alkaline phosphatase (ALP) expression and calcium accumulation were similar to non-printed MSCs after 3 weeks in osteogenic medium.

Keriquel et al. printed in situ MSCs on to a collagen/nanohydroxyapatite (nHA) disks placed cranial defects [ 82 ]. Compared to acellular collagen/nHA disks, the disks with the bioprinted MSC cells exhibited a larger bone volume after 2 months. Michael et al. printed 20 layers of keratinocytes on top of 20 layers of fibroblasts, situated on top of a carrier matrix, Matriderm ® that provided stability [ 21 ]. Keratinocytes developed into a stratified dense tissue in an in vivo study after 11 days implanted subcutaneously in mice, and demonstrated the potential for LAB in skin tissue regeneration.

LAB has the ability to position multiple cell types with a high degree of accuracy, with several studies demonstrating singular the capability of positioning a singular cell per droplet [ 29 , 81 , 83 ]. However, it is an expensive process to perform and suffers from low stability and scalability. It has shown great potential when combined with other biofabrication techniques [ 29 , 84 ].

5. Extrusion-Based Bioprinting

Extrusion-based printing is a pressure-driven technology. The bioink is extruded through a nozzle, driven either by pneumatic or mechanical pressure, and deposited in a predesigned structure, as depicted in Figure 5 [ 50 ]. The main advantage of extrusion bioprinting is the ability to print with very high cell densities [ 85 , 86 ]. Despite its versatility and benefits, it has some disadvantages when compared to other technologies. The resolution is very limited, as a minimum feature size is generally over 100 µm, which is a poorer resolution than that of other bioprinting techniques [ 87 ]. This could limit its application for certain soft tissue applications that require small pore sizes for an improved tissue response [ 11 , 86 , 88 ], however could still be applicable to hard tissues with size larger than 10 mm [ 35 , 86 ]. The pressure used for the extrusion of the material has the potential to alter the cell morphology and function, although several studies have reported [ 86 ]. Overall, before printing of the hydrogel can be performed a detailed study with different process parameters including viscosity, nozzle diameter and the accompanied shear stress has to be evaluated [ 89 , 90 ]. This fabrication technique uses highly viscous hydrogel and does not necessarily require any chemical additives for the curing of printed structure [ 86 ]. Rheological behavior of the hydrogel ink is very important for extrusion-based bioprinting. Hydrogels are mostly non-Newtonian fluids, meaning that their viscosity changes with shear rate. However, the more viscous the bioink, the higher the induced shear-stress during printing, resulting in higher cell apoptotic activity. An important phenomenon in non-Newtonian fluids is shear thinning, which is a drop of viscosity with an applied shear force. This has a direct impact on the print quality, enabling a plug-like flow to be established, providing greater control over starting and stopping the extrusion process [ 91 ]. Although low viscosities result in less dense networks that could allow for better cellular infiltration, too low viscosities will produce a structure that has a poor definition that will ultimately affect print fidelity.

Schematic of Extrusion-based Bioprinting; from left, pneumatic-based and right, mechanical-based. Struts are extruded via pneumatic or mechanical pressure through micro-nozzles. Extrusion-based techniques can produce structures with great mechanical properties and print fidelity.

A study conducted by Chung et al., observed the bioink properties and the printability of alginate-gelatin blends. Using Alg-Gel ink solutions, the printing of scaffolds from three different alginate concentrations (1, 2, and 4% w / v ) were compared. Both printed scaffolds using 2% Alg-Gel and 4% Alg-Gel demonstrated defined structures and maintained their ability to support optimal cell growth. The highly hydrated network structure permits the exchange of gases and nutrients [ 92 ]. When choosing a hydrogel to use as the base material, a trade-off must be made between rigidness and softness in order to have a strong supporting structure that allows for nutrient infiltration and the capability to encapsulate cells. High concentrations or crosslink densities are needed to keep a good printing fidelity, yet this limits cell migration. However, low concentrations usually have a poor printability and low mechanical properties. To improve the mechanical properties of the hydrogel, reinforcing fibers like PCL can be used [ 93 ]. Photopolymerization is emerging as a promising crosslinking reaction for bioprinting because it enables the rapid formation of hydrogels immediately after printing to maintain print fidelity through the incidence of light energy at appropriate wavelengths [ 1 , 94 , 95 , 96 , 97 ]. The printing resolution can also affected by the diffusion and fusion of the bioinks, which could be solved by reducing the extrusion rate or accelerating the moving speed. With good cell compatibility of the hydrogel material and the high printing quality with appropriate printing process parameters, the hydrogel deposition in the fabrication of tissues or organs can be obtained [ 98 ].

An important characteristic for the hydrogel is that it should maintain its mechanical properties after printing. During printing, the hydrogel is subjected to different forces. In nozzle based printing systems, such as with inkjet and extrusion-based techniques, high shear forces can break or disrupt the interlinking bonds of the hydrogel molecular network. This damage to the hydrogel crosslinking can cause a drop in viscosity and a reduction in print fidelity. To overcome this issue, research has been conducted into self-healing hydrogels [ 99 ]. A self-healing hydrogel can retain its printed shape due to its non-covalent reversible bonds [ 100 , 101 ]. An improved structure of hydrogels is a structure that has interpenetrating polymer networks (IPNs, which consist of 2 (or more) polymer networks; where one is crosslinked in the immediate presence of another [ 102 ]. The networks can be crosslinked simultaneously or sequentially, from heterogeneous or homogeneous materials. An example of IPNs made of heterogeneous materials is double network (DN) IPNs, which is fabricated in a 2-step polymerization process of rigid and soft hydrogels [ 103 ]. Biocompatible DNs have been successfully employed in cell encapsulation [ 104 ].

Cell survivability and function can also be negatively influenced by the extrusion process. In highly concentrated bioinks, shear stresses have the potential to cause cell apoptosis and a drop in the number of living cells [ 1 , 86 , 105 ]. Shear stress can also affect cell morphology and metabolic activity, as well as the adhesiveness of the cells to the substrate [ 86 ]. However, the overall cellular response is dependent upon cell type, as some cells are more resistant than others [ 86 ].

Extrusion printing can be regarded as a promising technology that allows the fabrication of organized constructs at clinically relevant sizes within a reasonable time frame. However, selection of biomaterial and bioink concentration is important for the survival of the cells during fabrication, as well as the maintenance of cell viability and functionality post-printing.

Lee et al. used an extrusion bioprinter to regenerate an ear formed of auricular cartilage and fat tissue [ 106 ]. The ear shaped scaffold was fabricated using chondrocytes and adipose-derived stromal cells, encapsulated in a hydrogel composed of PCL and poly(ethylene-glycol) (PEG). The bioprinted ear achieved a 95% cell viability [ 106 ]. The regeneration of the ear has been considered to be a challenge due to its complex structure and composition, which is difficult to replicate using traditional fabrication techniques.

Kundu et al. produced cartilage scaffolds by extruding alginate hydrogel onto PCL [ 107 ]. Scaffold were printed either with or without human inferior turbinate-tissue derived mesenchymal stromal cells (hTMSCs) within the alginate bioink. Better chondrogenic function was observed when the hTMSCs were encapsulated in alginate gel as well as an increase in extra cellular matrix (ECM) production without an adverse tissue response when implanted into the dorsal subcutaneous spaces of mice [ 107 ]. The encapsulation of the cells in alginate hydrogel showed negligible effects on the viability of the chondrocytes which addressed the formation and synthesis of cartilaginous ECM.

Pati et al. developed a hybrid scaffold combining PCL and decellularized extracellular matrix (dECM) [ 108 ]. The dECM bioink was loaded with stem cells derived from adipose, cartilage and heart tissues, and deposited into a PCL framework. It was observed that there was a cell-to-cell interconnectivity within 24 h and a cell and viability of 90% on day 7. This study shows the ability to print complex structures with appropriate material and cells, which can provide an optimized microenvironment that is conductive to the growth of 3D structured tissues.

Miri et al. demonstrated the possibility to create hierarchical cell laden structures to mimic multicellular tissues [ 26 ]. For in vitro studies, hydrogels including poly(ethylene glycol) diacrylate (PEGDA) and methacrylated gelatin (GelMA) loaded with NIH/3T3 fibroblasts and C2C12 skeletal muscle cells were printed into structures resembling musculoskeletal junctions, muscle strips and tumor angiogenesis. The prints retained interfaces and adequate proliferation rates after 3, 5 and 7 days in cell culture. PEGDA-framed chips that had a concentration-gradient of GelMA ranging from 5–15%, were implanted subcutaneously in rats. The result showed formation of the blood vessel network in the bioactive GelMA hydrogels, while the PEGDA served as the frame in the bioprinted multimaterial structure. This novel pneumatic-based process of creating microfluidic devices enabled the printing of different cell suspensions in order to achievemultimaterial devices.

Extrusion bioprinting is a promising technique to create biomimetic structures to replace tissues and organs. This technique was also efficient in creating microfluidic chips for research applications. Despite its great versatility and feasibility in vertical printing, extrusion-based bioprinting has a relatively limited resolution that does not allow for cell positioning, and requires an advanced hydrogel bioink that maintains cell viability as well as mechanical integrity which has led to the development and use of self-healing hydrogels as well as interpenetrating polymer networks.

6. Discussion

3D bioprinting is a relatively new aspect to tissue engineering and has opened the possibility of creating an unprecedented biomimicry, which could ultimately replace the current gold standard of autografts. Biomimicry, in form and function, has great significance in regenerative medicine, drug screening and understanding pathology [ 109 ]. In vitro applications have been used to assess pathological and toxicological conditions, as well as implant integration, and offers a methodology with a high-throughput [ 110 ]. Biomimetic microfluidic chips have great potential in replacing animal studies for drug and material screening.

Each bioprinting technique has different requirements for the bioink that can create diverse effects on the encapsulated cells. Inkjet bioprinting provides high resolution and accurate cell positioning. However, it requires the bioink to have a low concentration, which may result in poor structural integrity and inefficient cell encapsulation. This technique has shown great success in creating neural and skin tissues [ 32 , 59 ]. In skin tissue engineering, scaffolds fabricated using inkjet bioprinting have delivered better results when compared to a commercial graft Alpigraf ® to repair full thickness wounds in mice [ 37 ].

Stereolithography offers the possibility of printing cell-laden structures with the shortest fabrication time possible, hence limiting the exposure of the cells to non-physiological conditions. SLA fabrication does not inflict shear stresses upon the cells, unlike in nozzle based techniques, which have the potential to cause cell apoptosis. However, complex designs that include hollow structures (vessels, vasculature or ducts), can become blocked due to remnants of the precursor hydrogel within the printed pores [ 26 ]. Another problem with SLA is that surplus bioink is used as fabrication is performed in a vat. That vat is filled with a larger volume of biomaterial, cells and biomolecules than what is needed for the fabrication of the scaffold.

Extrusion-based printing is the most feasible technique in terms of vertical configuration, although has the lowest reported cell survival among all techniques. The low survivability is due to the shear stress that arises during printing. An important aspect of extrusion printing is its influence on the hydrogel during and after printing. Due to the high shear stresses induced during printing it is possible that the hydrogel could lose its structural integrity. This has led to the development of self-healing hydrogels, which regain their mechanical integrity after the application of shear [ 111 ]. Extrusion-based bioprinting has succeeded in creating complex tissue constructs and multi-material microfluidic devices [ 36 , 39 ].

A problem encountered by all techniques when using photopolymerization to harden the bioink, is the cytotoxicity of the photoinitiators used and the damage inflicted by UV (10–400 nm) or near-UV blue (400–490 nm) irradiation. However, alternatives to the use of UV light and the use of photoinitiators are under investigation. Visible light-sensitive photoinitiators have reported less cytotoxicity than the most commonly used UV-sensitive photoinitiators [ 63 ], as well as an enhanced print fidelity [ 78 ].

Post-fabrication, cell-laden scaffolds can be incubated in culture medium to ensure the attachment of cells [ 112 ]. Incubation for longer periods (21 days) has resulted in an increase of mechanical strength of the scaffolds due to tissue development [ 57 ]. Incubation can be static in cell culture or dynamic using bioreactors. Dynamic culturing can provide continuous infiltrating flow of medium and/or compressive/tensile loading, which is most beneficial for cartilage and bone tissue engineering [ 113 ].

Current research demonstrates the feasibility and efficiency of using more than one fabrication technique in the manufacturing process. Inkjet printing and LAB have the capability of accurate cell positioning with both of them having achieved the positioning of singular cells per droplet. However, inkjet printing is limited by its ability to produce a 3D architecture, whereas LAB only positions the bioink onto a prefabricated scaffold and is also associated with a high cost. In contrast, extrusion bioprinting has fast fabrication times for large 3D structures, yet has poor cell survivability. Therefore, by combining either inkjet bioprinting or LAB with extrusion printing could provide the ideal combination for producing a scaffold that has both physiologically relevant proportions as well as supports viable cells.

Research has already been implemented combining different printing techniques. In a study by Kim et al., a skin model was fabricated using an extrusion printer to create the main supporting structure and an inkjet printer was used to position dermal fibroblasts and epidermal keratinocytes within the scaffold [ 114 ]. The bioprinted scaffold formed dermal and epidermal layers after culturing. Another study combined extrusion printing with stereolithography to create a model for cancer research, where microfluidic devices were fabricated using a digital micro-mirror device and pneumatic extrusion, to understand tumor angiogenesis [ 26 ]. In situ applications, where the cell-laden biomaterial is directly deposited into the defect, are also being investigated for accelerated wound healing and bone regeneration, which have demonstrated improved results in comparison to non-cell containing grafts [ 53 , 65 , 66 ].

Finally, another aspect of bioprinting is its potential to provide prevascularization of the scaffolds. Accurate cell positioning in LAB and inkjet bioprinting techniques could enable a vasculature to be printed into a scaffold. Both techniques have shown promising results in positioning endothelial cells to induce angiogenesis [ 29 , 42 ]. Prevascularization is essential to avoid necrotic failure of the implantation. Other cell positioning research based on inkjet techniques shows great potential in constructing neural networks within large structures [ 32 ].

7. Conclusions

Additive manufacturing has been heavily applied to tissue engineering over the past decade. Bioprinting enables the production of scaffolds with a homogeneous distribution of cells throughout a scaffold. An organized distribution of different cell types can be positioned within the supporting material, mimicking tissues with multiple cell types or the interface between two tissues. While the choice of material and design impact the viability and proliferation of the printed cells, the different techniques have also shown variable cell activities post-fabrication. Bioprinting is still under development and has many bridges to cross before entering the clinical world, particularly as an in situ direct application. From this brief review, it is concluded that different applications require different fabrication techniques, depending on required resolution, speed, cost, the ability to print vertically etc. Future developments are now concentrating on the combining of techniques to work in a complementary fashion to optimize the process of creating tissue-mimicking structures.

Abbreviations

Author contributions.

S.A., S.R. and Z.I. conducted a literature review to provide the information of for this review article, Z.P.K., P.M.R. and M.B. wrote the article, R.S. and O.J. proof read the manuscript and helped with the final editing.

This research received no external funding

Conflicts of Interest

The authors declare no conflict of interest.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Review Article
• Open access
• Published: 25 October 2021

Augmented reality and virtual reality displays: emerging technologies and future perspectives

• Jianghao Xiong 1 ,
• En-Lin Hsiang 1 ,
• Ziqian He 1 ,
• Tao Zhan   ORCID: orcid.org/0000-0001-5511-6666 1 &
• Shin-Tson Wu   ORCID: orcid.org/0000-0002-0943-0440 1  

Light: Science & Applications volume  10 , Article number:  216 ( 2021 ) Cite this article

113k Accesses

431 Citations

36 Altmetric

Metrics details

• Liquid crystals

With rapid advances in high-speed communication and computation, augmented reality (AR) and virtual reality (VR) are emerging as next-generation display platforms for deeper human-digital interactions. Nonetheless, to simultaneously match the exceptional performance of human vision and keep the near-eye display module compact and lightweight imposes unprecedented challenges on optical engineering. Fortunately, recent progress in holographic optical elements (HOEs) and lithography-enabled devices provide innovative ways to tackle these obstacles in AR and VR that are otherwise difficult with traditional optics. In this review, we begin with introducing the basic structures of AR and VR headsets, and then describing the operation principles of various HOEs and lithography-enabled devices. Their properties are analyzed in detail, including strong selectivity on wavelength and incident angle, and multiplexing ability of volume HOEs, polarization dependency and active switching of liquid crystal HOEs, device fabrication, and properties of micro-LEDs (light-emitting diodes), and large design freedoms of metasurfaces. Afterwards, we discuss how these devices help enhance the AR and VR performance, with detailed description and analysis of some state-of-the-art architectures. Finally, we cast a perspective on potential developments and research directions of these photonic devices for future AR and VR displays.

Similar content being viewed by others

Advanced liquid crystal devices for augmented reality and virtual reality displays: principles and applications

Kun Yin, En-Lin Hsiang, … Shin-Tson Wu

Achromatic diffractive liquid-crystal optics for virtual reality displays

Zhenyi Luo, Yannanqi Li, … Shin-Tson Wu

Metasurface wavefront control for high-performance user-natural augmented reality waveguide glasses

Hyunpil Boo, Yoo Seung Lee, … Chee Wei Wong

Introduction

Recent advances in high-speed communication and miniature mobile computing platforms have escalated a strong demand for deeper human-digital interactions beyond traditional flat panel displays. Augmented reality (AR) and virtual reality (VR) headsets 1 , 2 are emerging as next-generation interactive displays with the ability to provide vivid three-dimensional (3D) visual experiences. Their useful applications include education, healthcare, engineering, and gaming, just to name a few 3 , 4 , 5 . VR embraces a total immersive experience, while AR promotes the interaction between user, digital contents, and real world, therefore displaying virtual images while remaining see-through capability. In terms of display performance, AR and VR face several common challenges to satisfy demanding human vision requirements, including field of view (FoV), eyebox, angular resolution, dynamic range, and correct depth cue, etc. Another pressing demand, although not directly related to optical performance, is ergonomics. To provide a user-friendly wearing experience, AR and VR should be lightweight and ideally have a compact, glasses-like form factor. The above-mentioned requirements, nonetheless, often entail several tradeoff relations with one another, which makes the design of high-performance AR/VR glasses/headsets particularly challenging.

In the 1990s, AR/VR experienced the first boom, which quickly subsided due to the lack of eligible hardware and digital content 6 . Over the past decade, the concept of immersive displays was revisited and received a new round of excitement. Emerging technologies like holography and lithography have greatly reshaped the AR/VR display systems. In this article, we firstly review the basic requirements of AR/VR displays and their associated challenges. Then, we briefly describe the properties of two emerging technologies: holographic optical elements (HOEs) and lithography-based devices (Fig. 1 ). Next, we separately introduce VR and AR systems because of their different device structures and requirements. For the immersive VR system, the major challenges and how these emerging technologies help mitigate the problems will be discussed. For the see-through AR system, we firstly review the present status of light engines and introduce some architectures for the optical combiners. Performance summaries on microdisplay light engines and optical combiners will be provided, that serve as a comprehensive overview of the current AR display systems.

The left side illustrates HOEs and lithography-based devices. The right side shows the challenges in VR and architectures in AR, and how the emerging technologies can be applied

Key parameters of AR and VR displays

AR and VR displays face several common challenges to satisfy the demanding human vision requirements, such as FoV, eyebox, angular resolution, dynamic range, and correct depth cue, etc. These requirements often exhibit tradeoffs with one another. Before diving into detailed relations, it is beneficial to review the basic definitions of the above-mentioned display parameters.

Definition of parameters

Taking a VR system (Fig. 2a ) as an example. The light emitting from the display module is projected to a FoV, which can be translated to the size of the image perceived by the viewer. For reference, human vision’s horizontal FoV can be as large as 160° for monocular vision and 120° for overlapped binocular vision 6 . The intersection area of ray bundles forms the exit pupil, which is usually correlated with another parameter called eyebox. The eyebox defines the region within which the whole image FoV can be viewed without vignetting. It therefore generally manifests a 3D geometry 7 , whose volume is strongly dependent on the exit pupil size. A larger eyebox offers more tolerance to accommodate the user’s diversified interpupillary distance (IPD) and wiggling of headset when in use. Angular resolution is defined by dividing the total resolution of the display panel by FoV, which measures the sharpness of a perceived image. For reference, a human visual acuity of 20/20 amounts to 1 arcmin angular resolution, or 60 pixels per degree (PPD), which is considered as a common goal for AR and VR displays. Another important feature of a 3D display is depth cue. Depth cue can be induced by displaying two separate images to the left eye and the right eye, which forms the vergence cue. But the fixed depth of the displayed image often mismatches with the actual depth of the intended 3D image, which leads to incorrect accommodation cues. This mismatch causes the so-called vergence-accommodation conflict (VAC), which will be discussed in detail later. One important observation is that the VAC issue may be more serious in AR than VR, because the image in an AR display is directly superimposed onto the real-world with correct depth cues. The image contrast is dependent on the display panel and stray light. To achieve a high dynamic range, the display panel should exhibit high brightness, low dark level, and more than 10-bits of gray levels. Nowadays, the display brightness of a typical VR headset is about 150–200 cd/m 2 (or nits).

a Schematic of a VR display defining FoV, exit pupil, eyebox, angular resolution, and accommodation cue mismatch. b Sketch of an AR display illustrating ACR

Figure 2b depicts a generic structure of an AR display. The definition of above parameters remains the same. One major difference is the influence of ambient light on the image contrast. For a see-through AR display, ambient contrast ratio (ACR) 8 is commonly used to quantify the image contrast:

where L on ( L off ) represents the on (off)-state luminance (unit: nit), L am is the ambient luminance, and T is the see-through transmittance. In general, ambient light is measured in illuminance (lux). For the convenience of comparison, we convert illuminance to luminance by dividing a factor of π, assuming the emission profile is Lambertian. In a normal living room, the illuminance is about 100 lux (i.e., L am  ≈ 30 nits), while in a typical office lighting condition, L am  ≈ 150 nits. For outdoors, on an overcast day, L am  ≈ 300 nits, and L am  ≈ 3000 nits on a sunny day. For AR displays, a minimum ACR should be 3:1 for recognizable images, 5:1 for adequate readability, and ≥10:1 for outstanding readability. To make a simple estimate without considering all the optical losses, to achieve ACR = 10:1 in a sunny day (~3000 nits), the display needs to deliver a brightness of at least 30,000 nits. This imposes big challenges in finding a high brightness microdisplay and designing a low loss optical combiner.

Tradeoffs and potential solutions

Next, let us briefly review the tradeoff relations mentioned earlier. To begin with, a larger FoV leads to a lower angular resolution for a given display resolution. In theory, to overcome this tradeoff only requires a high-resolution-display source, along with high-quality optics to support the corresponding modulation transfer function (MTF). To attain 60 PPD across 100° FoV requires a 6K resolution for each eye. This may be realizable in VR headsets because a large display panel, say 2–3 inches, can still accommodate a high resolution with acceptable manufacture cost. However, for a glasses-like wearable AR display, the conflict between small display size and the high solution becomes obvious as further shrinking the pixel size of a microdisplay is challenging.

To circumvent this issue, the concept of the foveated display is proposed 9 , 10 , 11 , 12 , 13 . The idea is based on that the human eye only has high visual acuity in the central fovea region, which accounts for about 10° FoV. If the high-resolution image is only projected to fovea while the peripheral image remains low resolution, then a microdisplay with 2K resolution can satisfy the need. Regarding the implementation method of foveated display, a straightforward way is to optically combine two display sources 9 , 10 , 11 : one for foveal and one for peripheral FoV. This approach can be regarded as spatial multiplexing of displays. Alternatively, time-multiplexing can also be adopted, by temporally changing the optical path to produce different magnification factors for the corresponding FoV 12 . Finally, another approach without multiplexing is to use a specially designed lens with intended distortion to achieve non-uniform resolution density 13 . Aside from the implementation of foveation, another great challenge is to dynamically steer the foveated region as the viewer’s eye moves. This task is strongly related to pupil steering, which will be discussed in detail later.

A larger eyebox or FoV usually decreases the image brightness, which often lowers the ACR. This is exactly the case for a waveguide AR system with exit pupil expansion (EPE) while operating under a strong ambient light. To improve ACR, one approach is to dynamically adjust the transmittance with a tunable dimmer 14 , 15 . Another solution is to directly boost the image brightness with a high luminance microdisplay and an efficient combiner optics. Details of this topic will be discussed in the light engine section.

Another tradeoff of FoV and eyebox in geometric optical systems results from the conservation of etendue (or optical invariant). To increase the system etendue requires a larger optics, which in turn compromises the form factor. Finally, to address the VAC issue, the display system needs to generate a proper accommodation cue, which often requires the modulation of image depth or wavefront, neither of which can be easily achieved in a traditional geometric optical system. While remarkable progresses have been made to adopt freeform surfaces 16 , 17 , 18 , to further advance AR and VR systems requires additional novel optics with a higher degree of freedom in structure design and light modulation. Moreover, the employed optics should be thin and lightweight. To mitigate the above-mentioned challenges, diffractive optics is a strong contender. Unlike geometric optics relying on curved surfaces to refract or reflect light, diffractive optics only requires a thin layer of several micrometers to establish efficient light diffractions. Two major types of diffractive optics are HOEs based on wavefront recording and manually written devices like surface relief gratings (SRGs) based on lithography. While SRGs have large design freedoms of local grating geometry, a recent publication 19 indicates the combination of HOE and freeform optics can also offer a great potential for arbitrary wavefront generation. Furthermore, the advances in lithography have also enabled optical metasurfaces beyond diffractive and refractive optics, and miniature display panels like micro-LED (light-emitting diode). These devices hold the potential to boost the performance of current AR/VR displays, while keeping a lightweight and compact form factor.

Formation and properties of HOEs

HOE generally refers to a recorded hologram that reproduces the original light wavefront. The concept of holography is proposed by Dennis Gabor 20 , which refers to the process of recording a wavefront in a medium (hologram) and later reconstructing it with a reference beam. Early holography uses intensity-sensitive recording materials like silver halide emulsion, dichromated gelatin, and photopolymer 21 . Among them, photopolymer stands out due to its easy fabrication and ability to capture high-fidelity patterns 22 , 23 . It has therefore found extensive applications like holographic data storage 23 and display 24 , 25 . Photopolymer HOEs (PPHOEs) have a relatively small refractive index modulation and therefore exhibits a strong selectivity on the wavelength and incident angle. Another feature of PPHOE is that several holograms can be recorded into a photopolymer film by consecutive exposures. Later, liquid-crystal holographic optical elements (LCHOEs) based on photoalignment polarization holography have also been developed 25 , 26 . Due to the inherent anisotropic property of liquid crystals, LCHOEs are extremely sensitive to the polarization state of the input light. This feature, combined with the polarization modulation ability of liquid crystal devices, offers a new possibility for dynamic wavefront modulation in display systems.

The formation of PPHOE is illustrated in Fig. 3a . When exposed to an interfering field with high-and-low intensity fringes, monomers tend to move toward bright fringes due to the higher local monomer-consumption rate. As a result, the density and refractive index is slightly larger in bright regions. Note the index modulation δ n here is defined as the difference between the maximum and minimum refractive indices, which may be twice the value in other definitions 27 . The index modulation δ n is typically in the range of 0–0.06. To understand the optical properties of PPHOE, we simulate a transmissive grating and a reflective grating using rigorous coupled-wave analysis (RCWA) 28 , 29 and plot the results in Fig. 3b . Details of grating configuration can be found in Table S1 . Here, the reason for only simulating gratings is that for a general HOE, the local region can be treated as a grating. The observation of gratings can therefore offer a general insight of HOEs. For a transmissive grating, its angular bandwidth (efficiency > 80%) is around 5° ( λ  = 550 nm), while the spectral band is relatively broad, with bandwidth around 175 nm (7° incidence). For a reflective grating, its spectral band is narrow, with bandwidth around 10 nm. The angular bandwidth varies with the wavelength, ranging from 2° to 20°. The strong selectivity of PPHOE on wavelength and incident angle is directly related to its small δ n , which can be adjusted by controlling the exposure dosage.

a Schematic of the formation of PPHOE. Simulated efficiency plots for b1 transmissive and b2 reflective PPHOEs. c Working principle of multiplexed PPHOE. d Formation and molecular configurations of LCHOEs. Simulated efficiency plots for e1 transmissive and e2 reflective LCHOEs. f Illustration of polarization dependency of LCHOEs

A distinctive feature of PPHOE is the ability to multiplex several holograms into one film sample. If the exposure dosage of a recording process is controlled so that the monomers are not completely depleted in the first exposure, the remaining monomers can continue to form another hologram in the following recording process. Because the total amount of monomer is fixed, there is usually an efficiency tradeoff between multiplexed holograms. The final film sample would exhibit the wavefront modulation functions of multiple holograms (Fig. 3c ).

Liquid crystals have also been used to form HOEs. LCHOEs can generally be categorized into volume-recording type and surface-alignment type. Volume-recording type LCHOEs are either based on early polarization holography recordings with azo-polymer 30 , 31 , or holographic polymer-dispersed liquid crystals (HPDLCs) 32 , 33 formed by liquid-crystal-doped photopolymer. Surface-alignment type LCHOEs are based on photoalignment polarization holography (PAPH) 34 . The first step is to record the desired polarization pattern in a thin photoalignment layer, and the second step is to use it to align the bulk liquid crystal 25 , 35 . Due to the simple fabrication process, high efficiency, and low scattering from liquid crystal’s self-assembly nature, surface-alignment type LCHOEs based on PAPH have recently attracted increasing interest in applications like near-eye displays. Here, we shall focus on this type of surface-alignment LCHOE and refer to it as LCHOE thereafter for simplicity.

The formation of LCHOEs is illustrated in Fig. 3d . The information of the wavefront and the local diffraction pattern is recorded in a thin photoalignment layer. The volume liquid crystal deposited on the photoalignment layer, depending on whether it is nematic liquid crystal or cholesteric liquid crystal (CLC), forms a transmissive or a reflective LCHOE. In a transmissive LCHOE, the bulk nematic liquid crystal molecules generally follow the pattern of the bottom alignment layer. The smallest allowable pattern period is governed by the liquid crystal distortion-free energy model, which predicts the pattern period should generally be larger than sample thickness 36 , 37 . This results in a maximum diffraction angle under 20°. On the other hand, in a reflective LCHOE 38 , 39 , the bulk CLC molecules form a stable helical structure, which is tilted to match the k -vector of the bottom pattern. The structure exhibits a very low distorted free energy 40 , 41 and can accommodate a pattern period that is small enough to diffract light into the total internal reflection (TIR) of a glass substrate.

The diffraction property of LCHOEs is shown in Fig. 3e . The maximum refractive index modulation of LCHOE is equal to the liquid crystal birefringence (Δ n ), which may vary from 0.04 to 0.5, depending on the molecular conjugation 42 , 43 . The birefringence used in our simulation is Δ n  = 0.15. Compared to PPHOEs, the angular and spectral bandwidths are significantly larger for both transmissive and reflective LCHOEs. For a transmissive LCHOE, its angular bandwidth is around 20° ( λ  = 550 nm), while the spectral bandwidth is around 300 nm (7° incidence). For a reflective LCHOE, its spectral bandwidth is around 80 nm and angular bandwidth could vary from 15° to 50°, depending on the wavelength.

The anisotropic nature of liquid crystal leads to LCHOE’s unique polarization-dependent response to an incident light. As depicted in Fig. 3f , for a transmissive LCHOE the accumulated phase is opposite for the conjugated left-handed circular polarization (LCP) and right-handed circular polarization (RCP) states, leading to reversed diffraction directions. For a reflective LCHOE, the polarization dependency is similar to that of a normal CLC. For the circular polarization with the same handedness as the helical structure of CLC, the diffraction is strong. For the opposite circular polarization, the diffraction is negligible.

Another distinctive property of liquid crystal is its dynamic response to an external voltage. The LC reorientation can be controlled with a relatively low voltage (<10 V rms ) and the response time is on the order of milliseconds, depending mainly on the LC viscosity and layer thickness. Methods to dynamically control LCHOEs can be categorized as active addressing and passive addressing, which can be achieved by either directly switching the LCHOE or modulating the polarization state with an active waveplate. Detailed addressing methods will be described in the VAC section.

Lithography-enabled devices

Lithography technologies are used to create arbitrary patterns on wafers, which lays the foundation of the modern integrated circuit industry 44 . Photolithography is suitable for mass production while electron/ion beam lithography is usually used to create photomask for photolithography or to write structures with nanometer-scale feature size. Recent advances in lithography have enabled engineered structures like optical metasurfaces 45 , SRGs 46 , as well as micro-LED displays 47 . Metasurfaces exhibit a remarkable design freedom by varying the shape of meta-atoms, which can be utilized to achieve novel functions like achromatic focus 48 and beam steering 49 . Similarly, SRGs also offer a large design freedom by manipulating the geometry of local grating regions to realize desired optical properties. On the other hand, micro-LED exhibits several unique features, such as ultrahigh peak brightness, small aperture ratio, excellent stability, and nanosecond response time, etc. As a result, micro-LED is a promising candidate for AR and VR systems for achieving high ACR and high frame rate for suppressing motion image blurs. In the following section, we will briefly review the fabrication and properties of micro-LEDs and optical modulators like metasurfaces and SRGs.

Fabrication and properties of micro-LEDs

LEDs with a chip size larger than 300 μm have been widely used in solid-state lighting and public information displays. Recently, micro-LEDs with chip sizes <5 μm have been demonstrated 50 . The first micro-LED disc with a diameter of about 12 µm was demonstrated in 2000 51 . After that, a single color (blue or green) LED microdisplay was demonstrated in 2012 52 . The high peak brightness, fast response time, true dark state, and long lifetime of micro-LEDs are attractive for display applications. Therefore, many companies have since released their micro-LED prototypes or products, ranging from large-size TVs to small-size microdisplays for AR/VR applications 53 , 54 . Here, we focus on micro-LEDs for near-eye display applications. Regarding the fabrication of micro-LEDs, through the metal-organic chemical vapor deposition (MOCVD) method, the AlGaInP epitaxial layer is grown on GaAs substrate for red LEDs, and GaN epitaxial layers on sapphire substrate for green and blue LEDs. Next, a photolithography process is applied to define the mesa and deposit electrodes. To drive the LED array, the fabricated micro-LEDs are transferred to a CMOS (complementary metal oxide semiconductor) driver board. For a small size (<2 inches) microdisplay used in AR or VR, the precision of the pick-and-place transfer process is hard to meet the high-resolution-density (>1000 pixel per inch) requirement. Thus, the main approach to assemble LED chips with driving circuits is flip-chip bonding 50 , 55 , 56 , 57 , as Fig. 4a depicts. In flip-chip bonding, the mesa and electrode pads should be defined and deposited before the transfer process, while metal bonding balls should be preprocessed on the CMOS substrate. After that, thermal-compression method is used to bond the two wafers together. However, due to the thermal mismatch of LED chip and driving board, as the pixel size decreases, the misalignment between the LED chip and the metal bonding ball on the CMOS substrate becomes serious. In addition, the common n-GaN layer may cause optical crosstalk between pixels, which degrades the image quality. To overcome these issues, the LED epitaxial layer can be firstly metal-bonded with the silicon driver board, followed by the photolithography process to define the LED mesas and electrodes. Without the need for an alignment process, the pixel size can be reduced to <5 µm 50 .

a Illustration of flip-chip bonding technology. b Simulated IQE-LED size relations for red and blue LEDs based on ABC model. c Comparison of EQE of different LED sizes with and without KOH and ALD side wall treatment. d Angular emission profiles of LEDs with different sizes. Metasurfaces based on e resonance-tuning, f non-resonance tuning and g combination of both. h Replication master and i replicated SRG based on nanoimprint lithography. Reproduced from a ref. 55 with permission from AIP Publishing, b ref. 61 with permission from PNAS, c ref. 66 with permission from IOP Publishing, d ref. 67 with permission from AIP Publishing, e ref. 69 with permission from OSA Publishing f ref. 48 with permission from AAAS g ref. 70 with permission from AAAS and h , i ref. 85 with permission from OSA Publishing

In addition to manufacturing process, the electrical and optical characteristics of LED also depend on the chip size. Generally, due to Shockley-Read-Hall (SRH) non-radiative recombination on the sidewall of active area, a smaller LED chip size results in a lower internal quantum efficiency (IQE), so that the peak IQE driving point will move toward a higher current density due to increased ratio of sidewall surface to active volume 58 , 59 , 60 . In addition, compared to the GaN-based green and blue LEDs, the AlGaInP-based red LEDs with a larger surface recombination and carrier diffusion length suffer a more severe efficiency drop 61 , 62 . Figure 4b shows the simulated result of IQE drop in relation with the LED chip size of blue and red LEDs based on ABC model 63 . To alleviate the efficiency drop caused by sidewall defects, depositing passivation materials by atomic layer deposition (ALD) or plasma enhanced chemical vapor deposition (PECVD) is proven to be helpful for both GaN and AlGaInP based LEDs 64 , 65 . In addition, applying KOH (Potassium hydroxide) treatment after ALD can further reduce the EQE drop of micro-LEDs 66 (Fig. 4c ). Small-size LEDs also exhibit some advantages, such as higher light extraction efficiency (LEE). Compared to an 100-µm LED, the LEE of a 2-µm LED increases from 12.2 to 25.1% 67 . Moreover, the radiation pattern of micro-LED is more directional than that of a large-size LED (Fig. 4d ). This helps to improve the lens collection efficiency in AR/VR display systems.

Metasurfaces and SGs

Thanks to the advances in lithography technology, low-loss dielectric metasurfaces working in the visible band have recently emerged as a platform for wavefront shaping 45 , 48 , 68 . They consist of an array of subwavelength-spaced structures with individually engineered wavelength-dependent polarization/phase/ amplitude response. In general, the light modulation mechanisms can be classified into resonant tuning 69 (Fig. 4e ), non-resonant tuning 48 (Fig. 4f ), and combination of both 70 (Fig. 4g ). In comparison with non-resonant tuning (based on geometric phase and/or dynamic propagation phase), the resonant tuning (such as Fabry–Pérot resonance, Mie resonance, etc.) is usually associated with a narrower operating bandwidth and a smaller out-of-plane aspect ratio (height/width) of nanostructures. As a result, they are easier to fabricate but more sensitive to fabrication tolerances. For both types, materials with a higher refractive index and lower absorption loss are beneficial to reduce the aspect ratio of nanostructure and improve the device efficiency. To this end, titanium dioxide (TiO 2 ) and gallium nitride (GaN) are the major choices for operating in the entire visible band 68 , 71 . While small-sized metasurfaces (diameter <1 mm) are usually fabricated via electron-beam lithography or focused ion beam milling in the labs, the ability of mass production is the key to their practical adoption. The deep ultraviolet (UV) photolithography has proven its feasibility for reproducing centimeter-size metalenses with decent imaging performance, while it requires multiple steps of etching 72 . Interestingly, the recently developed UV nanoimprint lithography based on a high-index nanocomposite only takes a single step and can obtain an aspect ratio larger than 10, which shows great promise for high-volume production 73 .

The arbitrary wavefront shaping capability and the thinness of the metasurfaces have aroused strong research interests in the development of novel AR/VR prototypes with improved performance. Lee et al. employed nanoimprint lithography to fabricate a centimeter-size, geometric-phase metalens eyepiece for full-color AR displays 74 . Through tailoring its polarization conversion efficiency and stacking with a circular polarizer, the virtual image can be superimposed with the surrounding scene. The large numerical aperture (NA~0.5) of the metalens eyepiece enables a wide FoV (>76°) that conventional optics are difficult to obtain. However, the geometric phase metalens is intrinsically a diffractive lens that also suffers from strong chromatic aberrations. To overcome this issue, an achromatic lens can be designed via simultaneously engineering the group delay and the group delay dispersion 75 , 76 , which will be described in detail later. Other novel and/or improved near-eye display architectures include metasurface-based contact lens-type AR 77 , achromatic metalens array enabled integral-imaging light field displays 78 , wide FoV lightguide AR with polarization-dependent metagratings 79 , and off-axis projection-type AR with an aberration-corrected metasurface combiner 80 , 81 , 82 . Nevertheless, from the existing AR/VR prototypes, metasurfaces still face a strong tradeoff between numerical aperture (for metalenses), chromatic aberration, monochromatic aberration, efficiency, aperture size, and fabrication complexity.

On the other hand, SRGs are diffractive gratings that have been researched for decades as input/output couplers of waveguides 83 , 84 . Their surface is composed of corrugated microstructures, and different shapes including binary, blazed, slanted, and even analogue can be designed. The parameters of the corrugated microstructures are determined by the target diffraction order, operation spectral bandwidth, and angular bandwidth. Compared to metasurfaces, SRGs have a much larger feature size and thus can be fabricated via UV photolithography and subsequent etching. They are usually replicated by nanoimprint lithography with appropriate heating and surface treatment. According to a report published a decade ago, SRGs with a height of 300 nm and a slant angle of up to 50° can be faithfully replicated with high yield and reproducibility 85 (Fig. 4g, h ).

Challenges and solutions of VR displays

The fully immersive nature of VR headset leads to a relatively fixed configuration where the display panel is placed in front of the viewer’s eye and an imaging optics is placed in-between. Regarding the system performance, although inadequate angular resolution still exists in some current VR headsets, the improvement of display panel resolution with advanced fabrication process is expected to solve this issue progressively. Therefore, in the following discussion, we will mainly focus on two major challenges: form factor and 3D cue generation.

Form factor

Compact and lightweight near-eye displays are essential for a comfortable user experience and therefore highly desirable in VR headsets. Current mainstream VR headsets usually have a considerably larger volume than eyeglasses, and most of the volume is just empty. This is because a certain distance is required between the display panel and the viewing optics, which is usually close to the focal length of the lens system as illustrated in Fig. 5a . Conventional VR headsets employ a transmissive lens with ~4 cm focal length to offer a large FoV and eyebox. Fresnel lenses are thinner than conventional ones, but the distance required between the lens and the panel does not change significantly. In addition, the diffraction artifacts and stray light caused by the Fresnel grooves can degrade the image quality, or MTF. Although the resolution density, quantified as pixel per inch (PPI), of current VR headsets is still limited, eventually Fresnel lens will not be an ideal solution when a high PPI display is available. The strong chromatic aberration of Fresnel singlet should also be compensated if a high-quality imaging system is preferred.

a Schematic of a basic VR optical configuration. b Achromatic metalens used as VR eyepiece. c VR based on curved display and lenslet array. d Basic working principle of a VR display based on pancake optics. e VR with pancake optics and Fresnel lens array. f VR with pancake optics based on purely HOEs. Reprinted from b ref. 87 under the Creative Commons Attribution 4.0 License. Adapted from c ref. 88 with permission from IEEE, e ref. 91 and f ref. 92 under the Creative Commons Attribution 4.0 License

It is tempting to replace the refractive elements with a single thin diffractive lens like a transmissive LCHOE. However, the diffractive nature of such a lens will result in serious color aberrations. Interestingly, metalenses can fulfil this objective without color issues. To understand how metalenses achieve achromatic focus, let us first take a glance at the general lens phase profile \(\Phi (\omega ,r)\) expanded as a Taylor series 75 :

where \(\varphi _0(\omega )\) is the phase at the lens center, \(F\left( \omega \right)\) is the focal length as a function of frequency ω , r is the radial coordinate, and \(\omega _0\) is the central operation frequency. To realize achromatic focus, \(\partial F{{{\mathrm{/}}}}\partial \omega\) should be zero. With a designed focal length, the group delay \(\partial \Phi (\omega ,r){{{\mathrm{/}}}}\partial \omega\) and the group delay dispersion \(\partial ^2\Phi (\omega ,r){{{\mathrm{/}}}}\partial \omega ^2\) can be determined, and \(\varphi _0(\omega )\) is an auxiliary degree of freedom of the phase profile design. In the design of an achromatic metalens, the group delay is a function of the radial coordinate and monotonically increases with the metalens radius. Many designs have proven that the group delay has a limited variation range 75 , 76 , 78 , 86 . According to Shrestha et al. 86 , there is an inevitable tradeoff between the maximum radius of the metalens, NA, and operation bandwidth. Thus, the reported achromatic metalenses at visible usually have limited lens aperture (e.g., diameter < 250 μm) and NA (e.g., <0.2). Such a tradeoff is undesirable in VR displays, as the eyepiece favors a large clear aperture (inch size) and a reasonably high NA (>0.3) to maintain a wide FoV and a reasonable eye relief 74 .

To overcome this limitation, Li et al. 87 proposed a novel zone lens method. Unlike the traditional phase Fresnel lens where the zones are determined by the phase reset, the new approach divides the zones by the group delay reset. In this way, the lens aperture and NA can be much enlarged, and the group delay limit is bypassed. A notable side effect of this design is the phase discontinuity at zone boundaries that will contribute to higher-order focusing. Therefore, significant efforts have been conducted to find the optimal zone transition locations and to minimize the phase discontinuities. Using this method, they have demonstrated an impressive 2-mm-diameter metalens with NA = 0.7 and nearly diffraction-limited focusing for the designed wavelengths (488, 532, 658 nm) (Fig. 5b ). Such a metalens consists of 681 zones and works for the visible band ranging from 470 to 670 nm, though the focusing efficiency is in the order of 10%. This is a great starting point for the achromatic metalens to be employed as a compact, chromatic-aberration-free eyepiece in near-eye displays. Future challenges are how to further increase the aperture size, correct the off-axis aberrations, and improve the optical efficiency.

Besides replacing the refractive lens with an achromatic metalens, another way to reduce system focal length without decreasing NA is to use a lenslet array 88 . As depicted in Fig. 5c , both the lenslet array and display panel adopt a curved structure. With the latest flexible OLED panel, the display can be easily curved in one dimension. The system exhibits a large diagonal FoV of 180° with an eyebox of 19 by 12 mm. The geometry of each lenslet is optimized separately to achieve an overall performance with high image quality and reduced distortions.

Aside from trying to shorten the system focal length, another way to reduce total track is to fold optical path. Recently, polarization-based folded lenses, also known as pancake optics, are under active development for VR applications 89 , 90 . Figure 5d depicts the structure of an exemplary singlet pancake VR lens system. The pancake lenses can offer better imaging performance with a compact form factor since there are more degrees of freedom in the design and the actual light path is folded thrice. By using a reflective surface with a positive power, the field curvature of positive refractive lenses can be compensated. Also, the reflective surface has no chromatic aberrations and it contributes considerable optical power to the system. Therefore, the optical power of refractive lenses can be smaller, resulting in an even weaker chromatic aberration. Compared to Fresnel lenses, the pancake lenses have smooth surfaces and much fewer diffraction artifacts and stray light. However, such a pancake lens design is not perfect either, whose major shortcoming is low light efficiency. With two incidences of light on the half mirror, the maximum system efficiency is limited to 25% for a polarized input and 12.5% for an unpolarized input light. Moreover, due to the existence of multiple surfaces in the system, stray light caused by surface reflections and polarization leakage may lead to apparent ghost images. As a result, the catadioptric pancake VR headset usually manifests a darker imagery and lower contrast than the corresponding dioptric VR.

Interestingly, the lenslet and pancake optics can be combined to further reduce the system form. Bang et al. 91 demonstrated a compact VR system with a pancake optics and a Fresnel lenslet array. The pancake optics serves to fold the optical path between the display panel and the lenslet array (Fig. 5e ). Another Fresnel lens is used to collect the light from the lenslet array. The system has a decent horizontal FoV of 102° and an eyebox of 8 mm. However, a certain degree of image discontinuity and crosstalk are still present, which can be improved with further optimizations on the Fresnel lens and the lenslet array.

One step further, replacing all conventional optics in catadioptric VR headset with holographic optics can make the whole system even thinner. Maimone and Wang demonstrated such a lightweight, high-resolution, and ultra-compact VR optical system using purely HOEs 92 . This holographic VR optics was made possible by combining several innovative optical components, including a reflective PPHOE, a reflective LCHOE, and a PPHOE-based directional backlight with laser illumination, as shown in Fig. 5f . Since all the optical power is provided by the HOEs with negligible weight and volume, the total physical thickness can be reduced to <10 mm. Also, unlike conventional bulk optics, the optical power of a HOE is independent of its thickness, only subject to the recording process. Another advantage of using holographic optical devices is that they can be engineered to offer distinct phase profiles for different wavelengths and angles of incidence, adding extra degrees of freedom in optical designs for better imaging performance. Although only a single-color backlight has been demonstrated, such a PPHOE has the potential to achieve full-color laser backlight with multiplexing ability. The PPHOE and LCHOE in the pancake optics can also be optimized at different wavelengths for achieving high-quality full-color images.

Vergence-accommodation conflict

Conventional VR displays suffer from VAC, which is a common issue for stereoscopic 3D displays 93 . In current VR display modules, the distance between the display panel and the viewing optics is fixed, which means the VR imagery is displayed at a single depth. However, the image contents are generated by parallax rendering in three dimensions, offering distinct images for two eyes. This approach offers a proper stimulus to vergence but completely ignores the accommodation cue, which leads to the well-known VAC that can cause an uncomfortable user experience. Since the beginning of this century, numerous methods have been proposed to solve this critical issue. Methods to produce accommodation cue include multifocal/varifocal display 94 , holographic display 95 , and integral imaging display 96 . Alternatively, elimination of accommodation cue using a Maxwellian-view display 93 also helps to mitigate the VAC. However, holographic displays and Maxwellian-view displays generally require a totally different optical architecture than current VR systems. They are therefore more suitable for AR displays, which will be discussed later. Integral imaging, on the other hand, has an inherent tradeoff between view number and resolution. For current VR headsets pursuing high resolution to match human visual acuity, it may not be an appealing solution. Therefore, multifocal/varifocal displays that rely on depth modulation is a relatively practical and effective solution for VR headsets. Regarding the working mechanism, multifocal displays present multiple images with different depths to imitate the original 3D scene. Varifocal displays, in contrast, only show one image at each time frame. The image depth matches the viewer’s vergence depth. Nonetheless, the pre-knowledge of the viewer’s vergence depth requires an additional eye-tracking module. Despite different operation principles, a varifocal display can often be converted to a multifocal display as long as the varifocal module has enough modulation bandwidth to support multiple depths in a time frame.

To achieve depth modulation in a VR system, traditional liquid lens 97 , 98 with tunable focus suffers from the small aperture and large aberrations. Alvarez lens 99 is another tunable-focus solution but it requires mechanical adjustment, which adds to system volume and complexity. In comparison, transmissive LCHOEs with polarization dependency can achieve focus adjustment with electronic driving. Its ultra-thinness also satisfies the requirement of small form factors in VR headsets. The diffractive behavior of transmissive LCHOEs is often interpreted by the mechanism of Pancharatnam-Berry phase (also known as geometric phase) 100 . They are therefore often called Pancharatnam-Berry optical elements (PBOEs). The corresponding lens component is referred as Pancharatnam-Berry lens (PBL).

Two main approaches are used to switch the focus of a PBL, active addressing and passive addressing. In active addressing, the PBL itself (made of LC) can be switched by an applied voltage (Fig. 6a ). The optical power of the liquid crystal PBLs can be turned-on and -off by controlling the voltage. Stacking multiple active PBLs can produce 2 N depths, where N is the number of PBLs. The drawback of using active PBLs, however, is the limited spectral bandwidth since their diffraction efficiency is usually optimized at a single wavelength. In passive addressing, the depth modulation is achieved through changing the polarization state of input light by a switchable half-wave plate (HWP) (Fig. 6b ). The focal length can therefore be switched thanks to the polarization sensitivity of PBLs. Although this approach has a slightly more complicated structure, the overall performance can be better than the active one, because the PBLs made of liquid crystal polymer can be designed to manifest high efficiency within the entire visible spectrum 101 , 102 .

Working principles of a depth switching PBL module based on a active addressing and b passive addressing. c A four-depth multifocal display based on time multiplexing. d A two-depth multifocal display based on polarization multiplexing. Reproduced from c ref. 103 with permission from OSA Publishing and d ref. 104 with permission from OSA Publishing

With the PBL module, multifocal displays can be built using time-multiplexing technique. Zhan et al. 103 demonstrated a four-depth multifocal display using two actively switchable liquid crystal PBLs (Fig. 6c ). The display is synchronized with the PBL module, which lowers the frame rate by the number of depths. Alternatively, multifocal displays can also be achieved by polarization-multiplexing, as demonstrated by Tan et al. 104 . The basic principle is to adjust the polarization state of local pixels so the image content on two focal planes of a PBL can be arbitrarily controlled (Fig. 6d ). The advantage of polarization multiplexing is that it does not sacrifice the frame rate, but it can only support two planes because only two orthogonal polarization states are available. Still, it can be combined with time-multiplexing to reduce the frame rate sacrifice by half. Naturally, varifocal displays can also be built with a PBL module. A fast-response 64-depth varifocal module with six PBLs has been demonstrated 105 .

The compact structure of PBL module leads to a natural solution of integrating it with above-mentioned pancake optics. A compact VR headset with dynamic depth modulation to solve VAC is therefore possible in practice. Still, due to the inherent diffractive nature of PBL, the PBL module face the issue of chromatic dispersion of focal length. To compensate for different focal depths for RGB colors may require additional digital corrections in image-rendering.

Architectures of AR displays

Unlike VR displays with a relatively fixed optical configuration, there exist a vast number of architectures in AR displays. Therefore, instead of following the narrative of tackling different challenges, a more appropriate way to review AR displays is to separately introduce each architecture and discuss its associated engineering challenges. An AR display usually consists of a light engine and an optical combiner. The light engine serves as display image source, while the combiner delivers the displayed images to viewer’s eye and in the meantime transmits the environment light. Some performance parameters like frame rate and power consumption are mainly determined by the light engine. Parameters like FoV, eyebox and MTF are primarily dependent on the combiner optics. Moreover, attributes like image brightness, overall efficiency, and form factor are influenced by both light engine and combiner. In this section, we will firstly discuss the light engine, where the latest advances in micro-LED on chip are reviewed and compared with existing microdisplay systems. Then, we will introduce two main types of combiners: free-space combiner and waveguide combiner.

Light engine

The light engine determines several essential properties of the AR system like image brightness, power consumption, frame rate, and basic etendue. Several types of microdisplays have been used in AR, including micro-LED, micro-organic-light-emitting-diodes (micro-OLED), liquid-crystal-on-silicon (LCoS), digital micromirror device (DMD), and laser beam scanning (LBS) based on micro-electromechanical system (MEMS). We will firstly describe the working principles of these devices and then analyze their performance. For those who are more interested in final performance parameters than details, Table 1 provides a comprehensive summary.

Working principles

Micro-LED and micro-OLED are self-emissive display devices. They are usually more compact than LCoS and DMD because no illumination optics is required. The fundamentally different material systems of LED and OLED lead to different approaches to achieve full-color displays. Due to the “green gap” in LEDs, red LEDs are manufactured on a different semiconductor material from green and blue LEDs. Therefore, how to achieve full-color display in high-resolution density microdisplays is quite a challenge for micro-LEDs. Among several solutions under research are two main approaches. The first is to combine three separate red, green and blue (RGB) micro-LED microdisplay panels 106 . Three single-color micro-LED microdisplays are manufactured separately through flip-chip transfer technology. Then, the projected images from three microdisplay panels are integrated by a trichroic prism (Fig. 7a ).

a RGB micro-LED microdisplays combined by a trichroic prism. b QD-based micro-LED microdisplay. c Micro-OLED display with 4032 PPI. Working principles of d LCoS, e DMD, and f MEMS-LBS display modules. Reprinted from a ref. 106 with permission from IEEE, b ref. 108 with permission from Chinese Laser Press, c ref. 121 with permission from Jon Wiley and Sons, d ref. 124 with permission from Spring Nature, e ref. 126 with permission from Springer and f ref. 128 under the Creative Commons Attribution 4.0 License

Another solution is to assemble color-conversion materials like quantum dot (QD) on top of blue or ultraviolet (UV) micro-LEDs 107 , 108 , 109 (Fig. 7b ). The quantum dot color filter (QDCF) on top of the micro-LED array is mainly fabricated by inkjet printing or photolithography 110 , 111 . However, the display performance of color-conversion micro-LED displays is restricted by the low color-conversion efficiency, blue light leakage, and color crosstalk. Extensive efforts have been conducted to improve the QD-micro-LED performance. To boost QD conversion efficiency, structure designs like nanoring 112 and nanohole 113 , 114 have been proposed, which utilize the Förster resonance energy transfer mechanism to transfer excessive excitons in the LED active region to QD. To prevent blue light leakage, methods using color filters or reflectors like distributed Bragg reflector (DBR) 115 and CLC film 116 on top of QDCF are proposed. Compared to color filters that absorb blue light, DBR and CLC film help recycle the leaked blue light to further excite QDs. Other methods to achieve full-color micro-LED display like vertically stacked RGB micro-LED array 61 , 117 , 118 and monolithic wavelength tunable nanowire LED 119 are also under investigation.

Micro-OLED displays can be generally categorized into RGB OLED and white OLED (WOLED). RGB OLED displays have separate sub-pixel structures and optical cavities, which resonate at the desirable wavelength in RGB channels, respectively. To deposit organic materials onto the separated RGB sub-pixels, a fine metal mask (FMM) that defines the deposition area is required. However, high-resolution RGB OLED microdisplays still face challenges due to the shadow effect during the deposition process through FMM. In order to break the limitation, a silicon nitride film with small shadow has been proposed as a mask for high-resolution deposition above 2000 PPI (9.3 µm) 120 .

WOLED displays use color filters to generate color images. Without the process of depositing patterned organic materials, a high-resolution density up to 4000 PPI has been achieved 121 (Fig. 7c ). However, compared to RGB OLED, the color filters in WOLED absorb about 70% of the emitted light, which limits the maximum brightness of the microdisplay. To improve the efficiency and peak brightness of WOLED microdisplays, in 2019 Sony proposed to apply newly designed cathodes (InZnO) and microlens arrays on OLED microdisplays, which increased the peak brightness from 1600 nits to 5000 nits 120 . In addition, OLEDWORKs has proposed a multi-stacked OLED 122 with optimized microcavities whose emission spectra match the transmission bands of the color filters. The multi-stacked OLED shows a higher luminous efficiency (cd/A), but also requires a higher driving voltage. Recently, by using meta-mirrors as bottom reflective anodes, patterned microcavities with more than 10,000 PPI have been obtained 123 . The high-resolution meta-mirrors generate different reflection phases in the RGB sub-pixels to achieve desirable resonant wavelengths. The narrow emission spectra from the microcavity help to reduce the loss from color filters or even eliminate the need of color filters.

LCoS and DMD are light-modulating displays that generate images by controlling the reflection of each pixel. For LCoS, the light modulation is achieved by manipulating the polarization state of output light through independently controlling the liquid crystal reorientation in each pixel 124 , 125 (Fig. 7d ). Both phase-only and amplitude modulators have been employed. DMD is an amplitude modulation device. The modulation is achieved through controlling the tilt angle of bi-stable micromirrors 126 (Fig. 7e ). To generate an image, both LCoS and DMD rely on the light illumination systems, with LED or laser as light source. For LCoS, the generation of color image can be realized either by RGB color filters on LCoS (with white LEDs) or color-sequential addressing (with RGB LEDs or lasers). However, LCoS requires a linearly polarized light source. For an unpolarized LED light source, usually, a polarization recycling system 127 is implemented to improve the optical efficiency. For a single-panel DMD, the color image is mainly obtained through color-sequential addressing. In addition, DMD does not require a polarized light so that it generally exhibits a higher efficiency than LCoS if an unpolarized light source is employed.

MEMS-based LBS 128 , 129 utilizes micromirrors to directly scan RGB laser beams to form two-dimensional (2D) images (Fig. 7f ). Different gray levels are achieved by pulse width modulation (PWM) of the employed laser diodes. In practice, 2D scanning can be achieved either through a 2D scanning mirror or two 1D scanning mirrors with an additional focusing lens after the first mirror. The small size of MEMS mirror offers a very attractive form factor. At the same time, the output image has a large depth-of-focus (DoF), which is ideal for projection displays. One shortcoming, though, is that the small system etendue often hinders its applications in some traditional display systems.

Comparison of light engine performance

There are several important parameters for a light engine, including image resolution, brightness, frame rate, contrast ratio, and form factor. The resolution requirement (>2K) is similar for all types of light engines. The improvement of resolution is usually accomplished through the manufacturing process. Thus, here we shall focus on other three parameters.

Image brightness usually refers to the measured luminance of a light-emitting object. This measurement, however, may not be accurate for a light engine as the light from engine only forms an intermediate image, which is not directly viewed by the user. On the other hand, to solely focus on the brightness of a light engine could be misleading for a wearable display system like AR. Nowadays, data projectors with thousands of lumens are available. But the power consumption is too high for a battery-powered wearable AR display. Therefore, a more appropriate way to evaluate a light engine’s brightness is to use luminous efficacy (lm/W) measured by dividing the final output luminous flux (lm) by the input electric power (W). For a self-emissive device like micro-LED or micro-OLED, the luminous efficacy is directly determined by the device itself. However, for LCoS and DMD, the overall luminous efficacy should take into consideration the light source luminous efficacy, the efficiency of illumination optics, and the efficiency of the employed spatial light modulator (SLM). For a MEMS LBS engine, the efficiency of MEMS mirror can be considered as unity so that the luminous efficacy basically equals to that of the employed laser sources.

As mentioned earlier, each light engine has a different scheme for generating color images. Therefore, we separately list luminous efficacy of each scheme for a more inclusive comparison. For micro-LEDs, the situation is more complicated because the EQE depends on the chip size. Based on previous studies 130 , 131 , 132 , 133 , we separately calculate the luminous efficacy for RGB micro-LEDs with chip size ≈ 20 µm. For the scheme of direct combination of RGB micro-LEDs, the luminous efficacy is around 5 lm/W. For QD-conversion with blue micro-LEDs, the luminous efficacy is around 10 lm/W with the assumption of 100% color conversion efficiency, which has been demonstrated using structure engineering 114 . For micro-OLEDs, the calculated luminous efficacy is about 4–8 lm/W 120 , 122 . However, the lifetime and EQE of blue OLED materials depend on the driving current. To continuously display an image with brightness higher than 10,000 nits may dramatically shorten the device lifetime. The reason we compare the light engine at 10,000 nits is that it is highly desirable to obtain 1000 nits for the displayed image in order to keep ACR>3:1 with a typical AR combiner whose optical efficiency is lower than 10%.

For an LCoS engine using a white LED as light source, the typical optical efficiency of the whole engine is around 10% 127 , 134 . Then the engine luminous efficacy is estimated to be 12 lm/W with a 120 lm/W white LED source. For a color sequential LCoS using RGB LEDs, the absorption loss from color filters is eliminated, but the luminous efficacy of RGB LED source is also decreased to about 30 lm/W due to lower efficiency of red and green LEDs and higher driving current 135 . Therefore, the final luminous efficacy of the color sequential LCoS engine is also around 10 lm/W. If RGB linearly polarized lasers are employed instead of LEDs, then the LCoS engine efficiency can be quite high due to the high degree of collimation. The luminous efficacy of RGB laser source is around 40 lm/W 136 . Therefore, the laser-based LCoS engine is estimated to have a luminous efficacy of 32 lm/W, assuming the engine optical efficiency is 80%. For a DMD engine with RGB LEDs as light source, the optical efficiency is around 50% 137 , 138 , which leads to a luminous efficacy of 15 lm/W. By switching to laser light sources, the situation is similar to LCoS, with the luminous efficacy of about 32 lm/W. Finally, for MEMS-based LBS engine, there is basically no loss from the optics so that the final luminous efficacy is 40 lm/W. Detailed calculations of luminous efficacy can be found in Supplementary Information .

Another aspect of a light engine is the frame rate, which determines the volume of information it can deliver in a unit time. A high volume of information is vital for the construction of a 3D light field to solve the VAC issue. For micro-LEDs, the device response time is around several nanoseconds, which allows for visible light communication with bandwidth up to 1.5 Gbit/s 139 . For an OLED microdisplay, a fast OLED with ~200 MHz bandwidth has been demonstrated 140 . Therefore, the limitation of frame rate is on the driving circuits for both micro-LED and OLED. Another fact concerning driving circuit is the tradeoff between resolution and frame rate as a higher resolution panel means more scanning lines in each frame. So far, an OLED display with 480 Hz frame rate has been demonstrated 141 . For an LCoS, the frame rate is mainly limited by the LC response time. Depending on the LC material used, the response time is around 1 ms for nematic LC or 200 µs for ferroelectric LC (FLC) 125 . Nematic LC allows analog driving, which accommodates gray levels, typically with 8-bit depth. FLC is bistable so that PWM is used to generate gray levels. DMD is also a binary device. The frame rate can reach 30 kHz, which is mainly constrained by the response time of micromirrors. For MEMS-based LBS, the frame rate is limited by the scanning frequency of MEMS mirrors. A frame rate of 60 Hz with around 1 K resolution already requires a resonance frequency of around 50 kHz, with a Q-factor up to 145,000 128 . A higher frame rate or resolution requires a higher Q-factor and larger laser modulation bandwidth, which may be challenging.

Form factor is another crucial aspect for the light engines of near-eye displays. For self-emissive displays, both micro-OLEDs and QD-based micro-LEDs can achieve full color with a single panel. Thus, they are quite compact. A micro-LED display with separate RGB panels naturally have a larger form factor. In applications requiring direct-view full-color panel, the extra combining optics may also increase the volume. It needs to be pointed out, however, that the combing optics may not be necessary for some applications like waveguide displays, because the EPE process results in system’s insensitivity to the spatial positions of input RGB images. Therefore, the form factor of using three RGB micro-LED panels is medium. For LCoS and DMD with RGB LEDs as light source, the form factor would be larger due to the illumination optics. Still, if a lower luminous efficacy can be accepted, then a smaller form factor can be achieved by using a simpler optics 142 . If RGB lasers are used, the collimation optics can be eliminated, which greatly reduces the form factor 143 . For MEMS-LBS, the form factor can be extremely compact due to the tiny size of MEMS mirror and laser module.

Finally, contrast ratio (CR) also plays an important role affecting the observed images 8 . Micro-LEDs and micro-OLEDs are self-emissive so that their CR can be >10 6 :1. For a laser beam scanner, its CR can also achieve 10 6 :1 because the laser can be turned off completely at dark state. On the other hand, LCoS and DMD are reflective displays, and their CR is around 2000:1 to 5000:1 144 , 145 . It is worth pointing out that the CR of a display engine plays a significant role only in the dark ambient. As the ambient brightness increases, the ACR is mainly governed by the display’s peak brightness, as previously discussed.

The performance parameters of different light engines are summarized in Table 1 . Micro-LEDs and micro-OLEDs have similar levels of luminous efficacy. But micro-OLEDs still face the burn-in and lifetime issue when driving at a high current, which hinders its use for a high-brightness image source to some extent. Micro-LEDs are still under active development and the improvement on luminous efficacy from maturing fabrication process could be expected. Both devices have nanosecond response time and can potentially achieve a high frame rate with a well-designed integrated circuit. The frame rate of the driving circuit ultimately determines the motion picture response time 146 . Their self-emissive feature also leads to a small form factor and high contrast ratio. LCoS and DMD engines have similar performance of luminous efficacy, form factor, and contrast ratio. In terms of light modulation, DMD can provide a higher 1-bit frame rate, while LCoS can offer both phase and amplitude modulations. MEMS-based LBS exhibits the highest luminous efficacy so far. It also exhibits an excellent form factor and contrast ratio, but the presently demonstrated 60-Hz frame rate (limited by the MEMS mirrors) could cause image flickering.

Free-space combiners

The term ‘free-space’ generally refers to the case when light is freely propagating in space, as opposed to a waveguide that traps light into TIRs. Regarding the combiner, it can be a partial mirror, as commonly used in AR systems based on traditional geometric optics. Alternatively, the combiner can also be a reflective HOE. The strong chromatic dispersion of HOE necessitates the use of a laser source, which usually leads to a Maxwellian-type system.

Traditional geometric designs

Several systems based on geometric optics are illustrated in Fig. 8 . The simplest design uses a single freeform half-mirror 6 , 147 to directly collimate the displayed images to the viewer’s eye (Fig. 8a ). This design can achieve a large FoV (up to 90°) 147 , but the limited design freedom with a single freeform surface leads to image distortions, also called pupil swim 6 . The placement of half-mirror also results in a relatively bulky form factor. Another design using so-called birdbath optics 6 , 148 is shown in Fig. 8b . Compared to the single-combiner design, birdbath design has an extra optics on the display side, which provides space for aberration correction. The integration of beam splitter provides a folded optical path, which reduces the form factor to some extent. Another way to fold optical path is to use a TIR-prism. Cheng et al. 149 designed a freeform TIR-prism combiner (Fig. 8c ) offering a diagonal FoV of 54° and exit pupil diameter of 8 mm. All the surfaces are freeform, which offer an excellent image quality. To cancel the optical power for the transmitted environmental light, a compensator is added to the TIR prism. The whole system has a well-balanced performance between FoV, eyebox, and form factor. To release the space in front of viewer’s eye, relay optics can be used to form an intermediate image near the combiner 150 , 151 , as illustrated in Fig. 8d . Although the design offers more optical surfaces for aberration correction, the extra lenses also add to system weight and form factor.

a Single freeform surface as the combiner. b Birdbath optics with a beam splitter and a half mirror. c Freeform TIR prism with a compensator. d Relay optics with a half mirror. Adapted from c ref. 149 with permission from OSA Publishing and d ref. 151 with permission from OSA Publishing

Regarding the approaches to solve the VAC issue, the most straightforward way is to integrate a tunable lens into the optical path, like a liquid lens 152 or Alvarez lens 99 , to form a varifocal system. Alternatively, integral imaging 153 , 154 can also be used, by replacing the original display panel with the central depth plane of an integral imaging module. The integral imaging can also be combined with varifocal approach to overcome the tradeoff between resolution and depth of field (DoF) 155 , 156 , 157 . However, the inherent tradeoff between resolution and view number still exists in this case.

Overall, AR displays based on traditional geometric optics have a relatively simple design with a decent FoV (~60°) and eyebox (8 mm) 158 . They also exhibit a reasonable efficiency. To measure the efficiency of an AR combiner, an appropriate measure is to divide the output luminance (unit: nit) by the input luminous flux (unit: lm), which we note as combiner efficiency. For a fixed input luminous flux, the output luminance, or image brightness, is related to the FoV and exit pupil of the combiner system. If we assume no light waste of the combiner system, then the maximum combiner efficiency for a typical diagonal FoV of 60° and exit pupil (10 mm square) is around 17,000 nit/lm (Eq. S2 ). To estimate the combiner efficiency of geometric combiners, we assume 50% of half-mirror transmittance and the efficiency of other optics to be 50%. Then the final combiner efficiency is about 4200 nit/lm, which is a high value in comparison with waveguide combiners. Nonetheless, to further shrink the system size or improve system performance ultimately encounters the etendue conservation issue. In addition, AR systems with traditional geometric optics is hard to achieve a configuration resembling normal flat glasses because the half-mirror has to be tilted to some extent.

Maxwellian-type systems

The Maxwellian view, proposed by James Clerk Maxwell (1860), refers to imaging a point light source in the eye pupil 159 . If the light beam is modulated in the imaging process, a corresponding image can be formed on the retina (Fig. 9a ). Because the point source is much smaller than the eye pupil, the image is always-in-focus on the retina irrespective of the eye lens’ focus. For applications in AR display, the point source is usually a laser with narrow angular and spectral bandwidths. LED light sources can also build a Maxwellian system, by adding an angular filtering module 160 . Regarding the combiner, although in theory a half-mirror can also be used, HOEs are generally preferred because they offer the off-axis configuration that places combiner in a similar position like eyeglasses. In addition, HOEs have a lower reflection of environment light, which provides a more natural appearance of the user behind the display.

a Schematic of the working principle of Maxwellian displays. Maxwellian displays based on b SLM and laser diode light source and c MEMS-LBS with a steering mirror as additional modulation method. Generation of depth cues by d computational digital holography and e scanning of steering mirror to produce multiple views. Adapted from b, d ref. 143 and c, e ref. 167 under the Creative Commons Attribution 4.0 License

To modulate the light, a SLM like LCoS or DMD can be placed in the light path, as shown in Fig. 9b . Alternatively, LBS system can also be used (Fig. 9c ), where the intensity modulation occurs in the laser diode itself. Besides the operation in a normal Maxwellian-view, both implementations offer additional degrees of freedom for light modulation.

For a SLM-based system, there are several options to arrange the SLM pixels 143 , 161 . Maimone et al. 143 demonstrated a Maxwellian AR display with two modes to offer a large-DoF Maxwellian-view, or a holographic view (Fig. 9d ), which is often referred as computer-generated holography (CGH) 162 . To show an always-in-focus image with a large DoF, the image can be directly displayed on an amplitude SLM, or using amplitude encoding for a phase-only SLM 163 . Alternatively, if a 3D scene with correct depth cues is to be presented, then optimization algorithms for CGH can be used to generate a hologram for the SLM. The generated holographic image exhibits the natural focus-and-blur effect like a real 3D object (Fig. 9d ). To better understand this feature, we need to again exploit the concept of etendue. The laser light source can be considered to have a very small etendue due to its excellent collimation. Therefore, the system etendue is provided by the SLM. The micron-sized pixel-pitch of SLM offers a certain maximum diffraction angle, which, multiplied by the SLM size, equals system etendue. By varying the display content on SLM, the final exit pupil size can be changed accordingly. In the case of a large-DoF Maxwellian view, the exit pupil size is small, accompanied by a large FoV. For the holographic display mode, the reduced DoF requires a larger exit pupil with dimension close to the eye pupil. But the FoV is reduced accordingly due to etendue conservation. Another commonly concerned issue with CGH is the computation time. To achieve a real-time CGH rendering flow with an excellent image quality is quite a challenge. Fortunately, with recent advances in algorithm 164 and the introduction of convolutional neural network (CNN) 165 , 166 , this issue is gradually solved with an encouraging pace. Lately, Liang et al. 166 demonstrated a real-time CGH synthesis pipeline with a high image quality. The pipeline comprises an efficient CNN model to generate a complex hologram from a 3D scene and an improved encoding algorithm to convert the complex hologram to a phase-only one. An impressive frame rate of 60 Hz has been achieved on a desktop computing unit.

For LBS-based system, the additional modulation can be achieved by integrating a steering module, as demonstrated by Jang et al. 167 . The steering mirror can shift the focal point (viewpoint) within the eye pupil, therefore effectively expanding the system etendue. When the steering process is fast and the image content is updated simultaneously, correct 3D cues can be generated, as shown in Fig. 9e . However, there exists a tradeoff between the number of viewpoint and the final image frame rate, because the total frames are equally divided into each viewpoint. To boost the frame rate of MEMS-LBS systems by the number of views (e.g., 3 by 3) may be challenging.

Maxwellian-type systems offer several advantages. The system efficiency is usually very high because nearly all the light is delivered into viewer’s eye. The system FoV is determined by the f /# of combiner and a large FoV (~80° in horizontal) can be achieved 143 . The issue of VAC can be mitigated with an infinite-DoF image that deprives accommodation cue, or completely solved by generating a true-3D scene as discussed above. Despite these advantages, one major weakness of Maxwellian-type system is the tiny exit pupil, or eyebox. A small deviation of eye pupil location from the viewpoint results in the complete disappearance of the image. Therefore, to expand eyebox is considered as one of the most important challenges in Maxwellian-type systems.

Pupil duplication and steering

Methods to expand eyebox can be generally categorized into pupil duplication 168 , 169 , 170 , 171 , 172 and pupil steering 9 , 13 , 167 , 173 . Pupil duplication simply generates multiple viewpoints to cover a large area. In contrast, pupil steering dynamically shifts the viewpoint position, depending on the pupil location. Before reviewing detailed implementations of these two methods, it is worth discussing some of their general features. The multiple viewpoints in pupil duplication usually mean to equally divide the total light intensity. In each time frame, however, it is preferable that only one viewpoint enters the user’s eye pupil to avoid ghost image. This requirement, therefore, results in a reduced total light efficiency, while also conditioning the viewpoint separation to be larger than the pupil diameter. In addition, the separation should not be too large to avoid gap between viewpoints. Considering that human pupil diameter changes in response to environment illuminance, the design of viewpoint separation needs special attention. Pupil steering, on the other hand, only produces one viewpoint at each time frame. It is therefore more light-efficient and free from ghost images. But to determine the viewpoint position requires the information of eye pupil location, which demands a real-time eye-tracking module 9 . Another observation is that pupil steering can accommodate multiple viewpoints by its nature. Therefore, a pupil steering system can often be easily converted to a pupil duplication system by simultaneously generating available viewpoints.

To generate multiple viewpoints, one can focus on modulating the incident light or the combiner. Recall that viewpoint is the image of light source. To duplicate or shift light source can achieve pupil duplication or steering accordingly, as illustrated in Fig. 10a . Several schemes of light modulation are depicted in Fig. 10b–e . An array of light sources can be generated with multiple laser diodes (Fig. 10b ). To turn on all or one of the sources achieves pupil duplication or steering. A light source array can also be produced by projecting light on an array-type PPHOE 168 (Fig. 10c ). Apart from direct adjustment of light sources, modulating light on the path can also effectively steer/duplicate the light sources. Using a mechanical steering mirror, the beam can be deflected 167 (Fig. 10d ), which equals to shifting the light source position. Other devices like a grating or beam splitter can also serve as ray deflector/splitter 170 , 171 (Fig. 10e ).

a Schematic of duplicating (or shift) viewpoint by modulation of incident light. Light modulation by b multiple laser diodes, c HOE lens array, d steering mirror and e grating or beam splitters. f Pupil duplication with multiplexed PPHOE. g Pupil steering with LCHOE. Reproduced from c ref. 168 under the Creative Commons Attribution 4.0 License, e ref. 169 with permission from OSA Publishing, f ref. 171 with permission from OSA Publishing and g ref. 173 with permission from OSA Publishing

Nonetheless, one problem of the light source duplication/shifting methods for pupil duplication/steering is that the aberrations in peripheral viewpoints are often serious 168 , 173 . The HOE combiner is usually recorded at one incident angle. For other incident angles with large deviations, considerable aberrations will occur, especially in the scenario of off-axis configuration. To solve this problem, the modulation can be focused on the combiner instead. While the mechanical shifting of combiner 9 can achieve continuous pupil steering, its integration into AR display with a small factor remains a challenge. Alternatively, the versatile functions of HOE offer possible solutions for combiner modulation. Kim and Park 169 demonstrated a pupil duplication system with multiplexed PPHOE (Fig. 10f ). Wavefronts of several viewpoints can be recorded into one PPHOE sample. Three viewpoints with a separation of 3 mm were achieved. However, a slight degree of ghost image and gap can be observed in the viewpoint transition. For a PPHOE to achieve pupil steering, the multiplexed PPHOE needs to record different focal points with different incident angles. If each hologram has no angular crosstalk, then with an additional device to change the light incident angle, the viewpoint can be steered. Alternatively, Xiong et al. 173 demonstrated a pupil steering system with LCHOEs in a simpler configuration (Fig. 10g ). The polarization-sensitive nature of LCHOE enables the controlling of which LCHOE to function with a polarization converter (PC). When the PC is off, the incident RCP light is focused by the right-handed LCHOE. When the PC is turned on, the RCP light is firstly converted to LCP light and passes through the right-handed LCHOE. Then it is focused by the left-handed LCHOE into another viewpoint. To add more viewpoints requires stacking more pairs of PC and LCHOE, which can be achieved in a compact manner with thin glass substrates. In addition, to realize pupil duplication only requires the stacking of multiple low-efficiency LCHOEs. For both PPHOEs and LCHOEs, because the hologram for each viewpoint is recorded independently, the aberrations can be eliminated.

Regarding the system performance, in theory the FoV is not limited and can reach a large value, such as 80° in horizontal direction 143 . The definition of eyebox is different from traditional imaging systems. For a single viewpoint, it has the same size as the eye pupil diameter. But due to the viewpoint steering/duplication capability, the total system eyebox can be expanded accordingly. The combiner efficiency for pupil steering systems can reach 47,000 nit/lm for a FoV of 80° by 80° and pupil diameter of 4 mm (Eq. S2 ). At such a high brightness level, eye safety could be a concern 174 . For a pupil duplication system, the combiner efficiency is decreased by the number of viewpoints. With a 4-by-4 viewpoint array, it can still reach 3000 nit/lm. Despite the potential gain of pupil duplication/steering, when considering the rotation of eyeball, the situation becomes much more complicated 175 . A perfect pupil steering system requires a 5D steering, which proposes a challenge for practical implementation.

Pin-light systems

Recently, another type of display in close relation with Maxwellian view called pin-light display 148 , 176 has been proposed. The general working principle of pin-light display is illustrated in Fig. 11a . Each pin-light source is a Maxwellian view with a large DoF. When the eye pupil is no longer placed near the source point as in Maxwellian view, each image source can only form an elemental view with a small FoV on retina. However, if the image source array is arranged in a proper form, the elemental views can be integrated together to form a large FoV. According to the specific optical architectures, pin-light display can take different forms of implementation. In the initial feasibility demonstration, Maimone et al. 176 used a side-lit waveguide plate as the point light source (Fig. 11b ). The light inside the waveguide plate is extracted by the etched divots, forming a pin-light source array. A transmissive SLM (LCD) is placed behind the waveguide plate to modulate the light intensity and form the image. The display has an impressive FoV of 110° thanks to the large scattering angle range. However, the direct placement of LCD before the eye brings issues of insufficient resolution density and diffraction of background light.

a Schematic drawing of the working principle of pin-light display. b Pin-light display utilizing a pin-light source and a transmissive SLM. c An example of pin-mirror display with a birdbath optics. d SWD system with LBS image source and off-axis lens array. Reprinted from b ref. 176 under the Creative Commons Attribution 4.0 License and d ref. 180 with permission from OSA Publishing

To avoid these issues, architectures using pin-mirrors 177 , 178 , 179 are proposed. In these systems, the final combiner is an array of tiny mirrors 178 , 179 or gratings 177 , in contrast to their counterparts using large-area combiners. An exemplary system with birdbath design is depicted in Fig. 11c . In this case, the pin-mirrors replace the original beam-splitter in the birdbath and can thus shrink the system volume, while at the same time providing large DoF pin-light images. Nonetheless, such a system may still face the etendue conservation issue. Meanwhile, the size of pin-mirror cannot be too small in order to prevent degradation of resolution density due to diffraction. Therefore, its influence on the see-through background should also be considered in the system design.

To overcome the etendue conservation and improve see-through quality, Xiong et al. 180 proposed another type of pin-light system exploiting the etendue expansion property of waveguide, which is also referred as scanning waveguide display (SWD). As illustrated in Fig. 11d , the system uses an LBS as the image source. The collimated scanned laser rays are trapped in the waveguide and encounter an array of off-axis lenses. Upon each encounter, the lens out-couples the laser rays and forms a pin-light source. SWD has the merits of good see-through quality and large etendue. A large FoV of 100° was demonstrated with the help of an ultra-low f /# lens array based on LCHOE. However, some issues like insufficient image resolution density and image non-uniformity remain to be overcome. To further improve the system may require optimization of Gaussian beam profile and additional EPE module 180 .

Overall, pin-light systems inherit the large DoF from Maxwellian view. With adequate number of pin-light sources, the FoV and eyebox can be expanded accordingly. Nonetheless, despite different forms of implementation, a common issue of pin-light system is the image uniformity. The overlapped region of elemental views has a higher light intensity than the non-overlapped region, which becomes even more complicated considering the dynamic change of pupil size. In theory, the displayed image can be pre-processed to compensate for the optical non-uniformity. But that would require knowledge of precise pupil location (and possibly size) and therefore an accurate eye-tracking module 176 . Regarding the system performance, pin-mirror systems modified from other free-space systems generally shares similar FoV and eyebox with original systems. The combiner efficiency may be lower due to the small size of pin-mirrors. SWD, on the other hand, shares the large FoV and DoF with Maxwellian view, and large eyebox with waveguide combiners. The combiner efficiency may also be lower due to the EPE process.

Waveguide combiner

Besides free-space combiners, another common architecture in AR displays is waveguide combiner. The term ‘waveguide’ indicates the light is trapped in a substrate by the TIR process. One distinctive feature of a waveguide combiner is the EPE process that effectively enlarges the system etendue. In the EPE process, a portion of the trapped light is repeatedly coupled out of the waveguide in each TIR. The effective eyebox is therefore enlarged. According to the features of couplers, we divide the waveguide combiners into two types: diffractive and achromatic, as described in the followings.

Diffractive waveguides

As the name implies, diffractive-type waveguides use diffractive elements as couplers. The in-coupler is usually a diffractive grating and the out-coupler in most cases is also a grating with the same period as the in-coupler, but it can also be an off-axis lens with a small curvature to generate image with finite depth. Three major diffractive couplers have been developed: SRGs, photopolymer gratings (PPGs), and liquid crystal gratings (grating-type LCHOE; also known as polarization volume gratings (PVGs)). Some general protocols for coupler design are that the in-coupler should have a relatively high efficiency and the out-coupler should have a uniform light output. A uniform light output usually requires a low-efficiency coupler, with extra degrees of freedom for local modulation of coupling efficiency. Both in-coupler and out-coupler should have an adequate angular bandwidth to accommodate a reasonable FoV. In addition, the out-coupler should also be optimized to avoid undesired diffractions, including the outward diffraction of TIR light and diffraction of environment light into user’s eyes, which are referred as light leakage and rainbow. Suppression of these unwanted diffractions should also be considered in the optimization process of waveguide design, along with performance parameters like efficiency and uniformity.

The basic working principles of diffractive waveguide-based AR systems are illustrated in Fig. 12 . For the SRG-based waveguides 6 , 8 (Fig. 12a ), the in-coupler can be a transmissive-type or a reflective-type 181 , 182 . The grating geometry can be optimized for coupling efficiency with a large degree of freedom 183 . For the out-coupler, a reflective SRG with a large slant angle to suppress the transmission orders is preferred 184 . In addition, a uniform light output usually requires a gradient efficiency distribution in order to compensate for the decreased light intensity in the out-coupling process. This can be achieved by varying the local grating configurations like height and duty cycle 6 . For the PPG-based waveguides 185 (Fig. 12b ), the small angular bandwidth of a high-efficiency transmissive PPG prohibits its use as in-coupler. Therefore, both in-coupler and out-coupler are usually reflective types. The gradient efficiency can be achieved by space-variant exposure to control the local index modulation 186 or local Bragg slant angle variation through freeform exposure 19 . Due to the relatively small angular bandwidth of PPG, to achieve a decent FoV usually requires stacking two 187 or three 188 PPGs together for a single color. The PVG-based waveguides 189 (Fig. 12c ) also prefer reflective PVGs as in-couplers because the transmissive PVGs are much more difficult to fabricate due to the LC alignment issue. In addition, the angular bandwidth of transmissive PVGs in Bragg regime is also not large enough to support a decent FoV 29 . For the out-coupler, the angular bandwidth of a single reflective PVG can usually support a reasonable FoV. To obtain a uniform light output, a polarization management layer 190 consisting of a LC layer with spatially variant orientations can be utilized. It offers an additional degree of freedom to control the polarization state of the TIR light. The diffraction efficiency can therefore be locally controlled due to the strong polarization sensitivity of PVG.

Schematics of waveguide combiners based on a SRGs, b PPGs and c PVGs. Reprinted from a ref. 85 with permission from OSA Publishing, b ref. 185 with permission from John Wiley and Sons and c ref. 189 with permission from OSA Publishing

The above discussion describes the basic working principle of 1D EPE. Nonetheless, for the 1D EPE to produce a large eyebox, the exit pupil in the unexpanded direction of the original image should be large. This proposes design challenges in light engines. Therefore, a 2D EPE is favored for practical applications. To extend EPE in two dimensions, two consecutive 1D EPEs can be used 191 , as depicted in Fig. 13a . The first 1D EPE occurs in the turning grating, where the light is duplicated in y direction and then turned into x direction. Then the light rays encounter the out-coupler and are expanded in x direction. To better understand the 2D EPE process, the k -vector diagram (Fig. 13b ) can be used. For the light propagating in air with wavenumber k 0 , its possible k -values in x and y directions ( k x and k y ) fall within the circle with radius k 0 . When the light is trapped into TIR, k x and k y are outside the circle with radius k 0 and inside the circle with radius nk 0 , where n is the refractive index of the substrate. k x and k y stay unchanged in the TIR process and are only changed in each diffraction process. The central red box in Fig. 13b indicates the possible k values within the system FoV. After the in-coupler, the k values are added by the grating k -vector, shifting the k values into TIR region. The turning grating then applies another k -vector and shifts the k values to near x -axis. Finally, the k values are shifted by the out-coupler and return to the free propagation region in air. One observation is that the size of red box is mostly limited by the width of TIR band. To accommodate a larger FoV, the outer boundary of TIR band needs to be expanded, which amounts to increasing waveguide refractive index. Another important fact is that when k x and k y are near the outer boundary, the uniformity of output light becomes worse. This is because the light propagation angle is near 90° in the waveguide. The spatial distance between two consecutive TIRs becomes so large that the out-coupled beams are spatially separated to an unacceptable degree. The range of possible k values for practical applications is therefore further shrunk due to this fact.

a Schematic of 2D EPE based on two consecutive 1D EPEs. Gray/black arrows indicate light in air/TIR. Black dots denote TIRs. b k-diagram of the two-1D-EPE scheme. c Schematic of 2D EPE with a 2D hexagonal grating d k-diagram of the 2D-grating scheme

Aside from two consecutive 1D EPEs, the 2D EPE can also be directly implemented with a 2D grating 192 . An example using a hexagonal grating is depicted in Fig. 13c . The hexagonal grating can provide k -vectors in six directions. In the k -diagram (Fig. 13d ), after the in-coupling, the k values are distributed into six regions due to multiple diffractions. The out-coupling occurs simultaneously with pupil expansion. Besides a concise out-coupler configuration, the 2D EPE scheme offers more degrees of design freedom than two 1D EPEs because the local grating parameters can be adjusted in a 2D manner. The higher design freedom has the potential to reach a better output light uniformity, but at the cost of a higher computation demand for optimization. Furthermore, the unslanted grating geometry usually leads to a large light leakage and possibly low efficiency. Adding slant to the geometry helps alleviate the issue, but the associated fabrication may be more challenging.

Finally, we discuss the generation of full-color images. One important issue to clarify is that although diffractive gratings are used here, the final image generally has no color dispersion even if we use a broadband light source like LED. This can be easily understood in the 1D EPE scheme. The in-coupler and out-coupler have opposite k -vectors, which cancels the color dispersion for each other. In the 2D EPE schemes, the k -vectors always form a closed loop from in-coupled light to out-coupled light, thus, the color dispersion also vanishes likewise. The issue of using a single waveguide for full-color images actually exists in the consideration of FoV and light uniformity. The breakup of propagation angles for different colors results in varied out-coupling situations for each color. To be more specific, if the red and the blue channels use the same in-coupler, the propagating angle for the red light is larger than that of the blue light. The red light in peripheral FoV is therefore easier to face the mentioned large-angle non-uniformity issue. To acquire a decent FoV and light uniformity, usually two or three layers of waveguides with different grating pitches are adopted.

Regarding the system performance, the eyebox is generally large enough (~10 mm) to accommodate different user’s IPD and alignment shift during operation. A parameter of significant concern for a waveguide combiner is its FoV. From the k -vector analysis, we can conclude the theoretical upper limit is determined by the waveguide refractive index. But the light/color uniformity also influences the effective FoV, over which the degradation of image quality becomes unacceptable. Current diffractive waveguide combiners generally achieve a FoV of about 50°. To further increase FoV, a straightforward method is to use a higher refractive index waveguide. Another is to tile FoV through direct stacking of multiple waveguides or using polarization-sensitive couplers 79 , 193 . As to the optical efficiency, a typical value for the diffractive waveguide combiner is around 50–200 nit/lm 6 , 189 . In addition, waveguide combiners adopting grating out-couplers generate an image with fixed depth at infinity. This leads to the VAC issue. To tackle VAC in waveguide architectures, the most practical way is to generate multiple depths and use the varifocal or multifocal driving scheme, similar to those mentioned in the VR systems. But to add more depths usually means to stack multiple layers of waveguides together 194 . Considering the additional waveguide layers for RGB colors, the final waveguide thickness would undoubtedly increase.

Other parameters special to waveguide includes light leakage, see-through ghost, and rainbow. Light leakage refers to out-coupled light that goes outwards to the environment, as depicted in Fig. 14a . Aside from decreased efficiency, the leakage also brings drawback of unnatural “bright-eye” appearance of the user and privacy issue. Optimization of the grating structure like geometry of SRG may reduce the leakage. See-through ghost is formed by consecutive in-coupling and out-couplings caused by the out-coupler grating, as sketched in Fig. 14b , After the process, a real object with finite depth may produce a ghost image with shift in both FoV and depth. Generally, an out-coupler with higher efficiency suffers more see-through ghost. Rainbow is caused by the diffraction of environment light into user’s eye, as sketched in Fig. 14c . The color dispersion in this case will occur because there is no cancellation of k -vector. Using the k -diagram, we can obtain a deeper insight into the formation of rainbow. Here, we take the EPE structure in Fig. 13a as an example. As depicted in Fig. 14d , after diffractions by the turning grating and the out-coupler grating, the k values are distributed in two circles that shift from the origin by the grating k -vectors. Some diffracted light can enter the see-through FoV and form rainbow. To reduce rainbow, a straightforward way is to use a higher index substrate. With a higher refractive index, the outer boundary of k diagram is expanded, which can accommodate larger grating k -vectors. The enlarged k -vectors would therefore “push” these two circles outwards, leading to a decreased overlapping region with the see-through FoV. Alternatively, an optimized grating structure would also help reduce the rainbow effect by suppressing the unwanted diffraction.

Sketches of formations of a light leakage, b see-through ghost and c rainbow. d Analysis of rainbow formation with k-diagram

Achromatic waveguide

Achromatic waveguide combiners use achromatic elements as couplers. It has the advantage of realizing full-color image with a single waveguide. A typical example of achromatic element is a mirror. The waveguide with partial mirrors as out-coupler is often referred as geometric waveguide 6 , 195 , as depicted in Fig. 15a . The in-coupler in this case is usually a prism to avoid unnecessary color dispersion if using diffractive elements otherwise. The mirrors couple out TIR light consecutively to produce a large eyebox, similarly in a diffractive waveguide. Thanks to the excellent optical property of mirrors, the geometric waveguide usually exhibits a superior image regarding MTF and color uniformity to its diffractive counterparts. Still, the spatially discontinuous configuration of mirrors also results in gaps in eyebox, which may be alleviated by using a dual-layer structure 196 . Wang et al. designed a geometric waveguide display with five partial mirrors (Fig. 15b ). It exhibits a remarkable FoV of 50° by 30° (Fig. 15c ) and an exit pupil of 4 mm with a 1D EPE. To achieve 2D EPE, similar architectures in Fig. 13a can be used by integrating a turning mirror array as the first 1D EPE module 197 . Unfortunately, the k -vector diagrams in Fig. 13b, d cannot be used here because the k values in x-y plane no longer conserve in the in-coupling and out-coupling processes. But some general conclusions remain valid, like a higher refractive index leading to a larger FoV and gradient out-coupling efficiency improving light uniformity.

a Schematic of the system configuration. b Geometric waveguide with five partial mirrors. c Image photos demonstrating system FoV. Adapted from b , c ref. 195 with permission from OSA Publishing

The fabrication process of geometric waveguide involves coating mirrors on cut-apart pieces and integrating them back together, which may result in a high cost, especially for the 2D EPE architecture. Another way to implement an achromatic coupler is to use multiplexed PPHOE 198 , 199 to mimic the behavior of a tilted mirror (Fig. 16a ). To understand the working principle, we can use the diagram in Fig. 16b . The law of reflection states the angle of reflection equals to the angle of incidence. If we translate this behavior to k -vector language, it means the mirror can apply any length of k -vector along its surface normal direction. The k -vector length of the reflected light is always equal to that of the incident light. This puts a condition that the k -vector triangle is isosceles. With a simple geometric deduction, it can be easily observed this leads to the law of reflection. The behavior of a general grating, however, is very different. For simplicity we only consider the main diffraction order. The grating can only apply a k -vector with fixed k x due to the basic diffraction law. For the light with a different incident angle, it needs to apply different k z to produce a diffracted light with equal k -vector length as the incident light. For a grating with a broad angular bandwidth like SRG, the range of k z is wide, forming a lengthy vertical line in Fig. 16b . For a PPG with a narrow angular bandwidth, the line is short and resembles a dot. If multiple of these tiny dots are distributed along the oblique line corresponding to a mirror, then the final multiplexed PPGs can imitate the behavior of a tilted mirror. Such a PPHOE is sometimes referred as a skew-mirror 198 . In theory, to better imitate the mirror, a lot of multiplexed PPGs is preferred, while each PPG has a small index modulation δn . But this proposes a bigger challenge in device fabrication. Recently, Utsugi et al. demonstrated an impressive skew-mirror waveguide based on 54 multiplexed PPGs (Fig. 16c, d ). The display exhibits an effective FoV of 35° by 36°. In the peripheral FoV, there still exists some non-uniformity (Fig. 16e ) due to the out-coupling gap, which is an inherent feature of the flat-type out-couplers.

a System configuration. b Diagram demonstrating how multiplexed PPGs resemble the behavior of a mirror. Photos showing c the system and d image. e Picture demonstrating effective system FoV. Adapted from c – e ref. 199 with permission from ITE

Finally, it is worth mentioning that metasurfaces are also promising to deliver achromatic gratings 200 , 201 for waveguide couplers ascribed to their versatile wavefront shaping capability. The mechanism of the achromatic gratings is similar to that of the achromatic lenses as previously discussed. However, the current development of achromatic metagratings is still in its infancy. Much effort is needed to improve the optical efficiency for in-coupling, control the higher diffraction orders for eliminating ghost images, and enable a large size design for EPE.

Generally, achromatic waveguide combiners exhibit a comparable FoV and eyebox with diffractive combiners, but with a higher efficiency. For a partial-mirror combiner, its combiner efficiency is around 650 nit/lm 197 (2D EPE). For a skew-mirror combiner, although the efficiency of multiplexed PPHOE is relatively low (~1.5%) 199 , the final combiner efficiency of the 1D EPE system is still high (>3000 nit/lm) due to multiple out-couplings.

Table 2 summarizes the performance of different AR combiners. When combing the luminous efficacy in Table 1 and the combiner efficiency in Table 2 , we can have a comprehensive estimate of the total luminance efficiency (nit/W) for different types of systems. Generally, Maxwellian-type combiners with pupil steering have the highest luminance efficiency when partnered with laser-based light engines like laser-backlit LCoS/DMD or MEM-LBS. Geometric optical combiners have well-balanced image performances, but to further shrink the system size remains a challenge. Diffractive waveguides have a relatively low combiner efficiency, which can be remedied by an efficient light engine like MEMS-LBS. Further development of coupler and EPE scheme would also improve the system efficiency and FoV. Achromatic waveguides have a decent combiner efficiency. The single-layer design also enables a smaller form factor. With advances in fabrication process, it may become a strong contender to presently widely used diffractive waveguides.

Conclusions and perspectives

VR and AR are endowed with a high expectation to revolutionize the way we interact with digital world. Accompanied with the expectation are the engineering challenges to squeeze a high-performance display system into a tightly packed module for daily wearing. Although the etendue conservation constitutes a great obstacle on the path, remarkable progresses with innovative optics and photonics continue to take place. Ultra-thin optical elements like PPHOEs and LCHOEs provide alternative solutions to traditional optics. Their unique features of multiplexing capability and polarization dependency further expand the possibility of novel wavefront modulations. At the same time, nanoscale-engineered metasurfaces/SRGs provide large design freedoms to achieve novel functions beyond conventional geometric optical devices. Newly emerged micro-LEDs open an opportunity for compact microdisplays with high peak brightness and good stability. Further advances on device engineering and manufacturing process are expected to boost the performance of metasurfaces/SRGs and micro-LEDs for AR and VR applications.

Data availability

All data needed to evaluate the conclusions in the paper are present in the paper. Additional data related to this paper may be requested from the authors.

Cakmakci, O. & Rolland, J. Head-worn displays: a review. J. Disp. Technol. 2 , 199–216 (2006).

Article   ADS   Google Scholar  

Zhan, T. et al. Augmented reality and virtual reality displays: perspectives and challenges. iScience 23 , 101397 (2020).

Rendon, A. A. et al. The effect of virtual reality gaming on dynamic balance in older adults. Age Ageing 41 , 549–552 (2012).

Article   Google Scholar  

Choi, S., Jung, K. & Noh, S. D. Virtual reality applications in manufacturing industries: past research, present findings, and future directions. Concurrent Eng. 23 , 40–63 (2015).

Li, X. et al. A critical review of virtual and augmented reality (VR/AR) applications in construction safety. Autom. Constr. 86 , 150–162 (2018).

Kress, B. C. Optical Architectures for Augmented-, Virtual-, and Mixed-Reality Headsets (Bellingham: SPIE Press, 2020).

Cholewiak, S. A. et al. A perceptual eyebox for near-eye displays. Opt. Express 28 , 38008–38028 (2020).

Lee, Y. H., Zhan, T. & Wu, S. T. Prospects and challenges in augmented reality displays. Virtual Real. Intell. Hardw. 1 , 10–20 (2019).

Kim, J. et al. Foveated AR: dynamically-foveated augmented reality display. ACM Trans. Graph. 38 , 99 (2019).

Tan, G. J. et al. Foveated imaging for near-eye displays. Opt. Express 26 , 25076–25085 (2018).

Lee, S. et al. Foveated near-eye display for mixed reality using liquid crystal photonics. Sci. Rep. 10 , 16127 (2020).

Yoo, C. et al. Foveated display system based on a doublet geometric phase lens. Opt. Express 28 , 23690–23702 (2020).

Akşit, K. et al. Manufacturing application-driven foveated near-eye displays. IEEE Trans. Vis. Computer Graph. 25 , 1928–1939 (2019).

Zhu, R. D. et al. High-ambient-contrast augmented reality with a tunable transmittance liquid crystal film and a functional reflective polarizer. J. Soc. Inf. Disp. 24 , 229–233 (2016).

Lincoln, P. et al. Scene-adaptive high dynamic range display for low latency augmented reality. In Proc. 21st ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games . (ACM, San Francisco, CA, 2017).

Duerr, F. & Thienpont, H. Freeform imaging systems: fermat’s principle unlocks “first time right” design. Light.: Sci. Appl. 10 , 95 (2021).

Bauer, A., Schiesser, E. M. & Rolland, J. P. Starting geometry creation and design method for freeform optics. Nat. Commun. 9 , 1756 (2018).

Rolland, J. P. et al. Freeform optics for imaging. Optica 8 , 161–176 (2021).

Jang, C. et al. Design and fabrication of freeform holographic optical elements. ACM Trans. Graph. 39 , 184 (2020).

Gabor, D. A new microscopic principle. Nature 161 , 777–778 (1948).

Kostuk, R. K. Holography: Principles and Applications (Boca Raton: CRC Press, 2019).

Lawrence, J. R., O'Neill, F. T. & Sheridan, J. T. Photopolymer holographic recording material. Optik 112 , 449–463 (2001).

Guo, J. X., Gleeson, M. R. & Sheridan, J. T. A review of the optimisation of photopolymer materials for holographic data storage. Phys. Res. Int. 2012 , 803439 (2012).

Jang, C. et al. Recent progress in see-through three-dimensional displays using holographic optical elements [Invited]. Appl. Opt. 55 , A71–A85 (2016).

Xiong, J. H. et al. Holographic optical elements for augmented reality: principles, present status, and future perspectives. Adv. Photonics Res. 2 , 2000049 (2021).

Tabiryan, N. V. et al. Advances in transparent planar optics: enabling large aperture, ultrathin lenses. Adv. Optical Mater. 9 , 2001692 (2021).

Zanutta, A. et al. Photopolymeric films with highly tunable refractive index modulation for high precision diffractive optics. Optical Mater. Express 6 , 252–263 (2016).

Moharam, M. G. & Gaylord, T. K. Rigorous coupled-wave analysis of planar-grating diffraction. J. Optical Soc. Am. 71 , 811–818 (1981).

Xiong, J. H. & Wu, S. T. Rigorous coupled-wave analysis of liquid crystal polarization gratings. Opt. Express 28 , 35960–35971 (2020).

Xie, S., Natansohn, A. & Rochon, P. Recent developments in aromatic azo polymers research. Chem. Mater. 5 , 403–411 (1993).

Shishido, A. Rewritable holograms based on azobenzene-containing liquid-crystalline polymers. Polym. J. 42 , 525–533 (2010).

Bunning, T. J. et al. Holographic polymer-dispersed liquid crystals (H-PDLCs). Annu. Rev. Mater. Sci. 30 , 83–115 (2000).

Liu, Y. J. & Sun, X. W. Holographic polymer-dispersed liquid crystals: materials, formation, and applications. Adv. Optoelectron. 2008 , 684349 (2008).

Xiong, J. H. & Wu, S. T. Planar liquid crystal polarization optics for augmented reality and virtual reality: from fundamentals to applications. eLight 1 , 3 (2021).

Yaroshchuk, O. & Reznikov, Y. Photoalignment of liquid crystals: basics and current trends. J. Mater. Chem. 22 , 286–300 (2012).

Sarkissian, H. et al. Periodically aligned liquid crystal: potential application for projection displays. Mol. Cryst. Liq. Cryst. 451 , 1–19 (2006).

Komanduri, R. K. & Escuti, M. J. Elastic continuum analysis of the liquid crystal polarization grating. Phys. Rev. E 76 , 021701 (2007).

Kobashi, J., Yoshida, H. & Ozaki, M. Planar optics with patterned chiral liquid crystals. Nat. Photonics 10 , 389–392 (2016).

Lee, Y. H., Yin, K. & Wu, S. T. Reflective polarization volume gratings for high efficiency waveguide-coupling augmented reality displays. Opt. Express 25 , 27008–27014 (2017).

Lee, Y. H., He, Z. Q. & Wu, S. T. Optical properties of reflective liquid crystal polarization volume gratings. J. Optical Soc. Am. B 36 , D9–D12 (2019).

Xiong, J. H., Chen, R. & Wu, S. T. Device simulation of liquid crystal polarization gratings. Opt. Express 27 , 18102–18112 (2019).

Czapla, A. et al. Long-period fiber gratings with low-birefringence liquid crystal. Mol. Cryst. Liq. Cryst. 502 , 65–76 (2009).

Dąbrowski, R., Kula, P. & Herman, J. High birefringence liquid crystals. Crystals 3 , 443–482 (2013).

Mack, C. Fundamental Principles of Optical Lithography: The Science of Microfabrication (Chichester: John Wiley & Sons, 2007).

Genevet, P. et al. Recent advances in planar optics: from plasmonic to dielectric metasurfaces. Optica 4 , 139–152 (2017).

Guo, L. J. Nanoimprint lithography: methods and material requirements. Adv. Mater. 19 , 495–513 (2007).

Park, J. et al. Electrically driven mid-submicrometre pixelation of InGaN micro-light-emitting diode displays for augmented-reality glasses. Nat. Photonics 15 , 449–455 (2021).

Khorasaninejad, M. et al. Metalenses at visible wavelengths: diffraction-limited focusing and subwavelength resolution imaging. Science 352 , 1190–1194 (2016).

Li, S. Q. et al. Phase-only transmissive spatial light modulator based on tunable dielectric metasurface. Science 364 , 1087–1090 (2019).

Liang, K. L. et al. Advances in color-converted micro-LED arrays. Jpn. J. Appl. Phys. 60 , SA0802 (2020).

Jin, S. X. et al. GaN microdisk light emitting diodes. Appl. Phys. Lett. 76 , 631–633 (2000).

Day, J. et al. Full-scale self-emissive blue and green microdisplays based on GaN micro-LED arrays. In Proc. SPIE 8268, Quantum Sensing and Nanophotonic Devices IX (SPIE, San Francisco, California, United States, 2012).

Huang, Y. G. et al. Mini-LED, micro-LED and OLED displays: present status and future perspectives. Light.: Sci. Appl. 9 , 105 (2020).

Parbrook, P. J. et al. Micro-light emitting diode: from chips to applications. Laser Photonics Rev. 15 , 2000133 (2021).

Day, J. et al. III-Nitride full-scale high-resolution microdisplays. Appl. Phys. Lett. 99 , 031116 (2011).

Liu, Z. J. et al. 360 PPI flip-chip mounted active matrix addressable light emitting diode on silicon (LEDoS) micro-displays. J. Disp. Technol. 9 , 678–682 (2013).

Zhang, L. et al. Wafer-scale monolithic hybrid integration of Si-based IC and III–V epi-layers—A mass manufacturable approach for active matrix micro-LED micro-displays. J. Soc. Inf. Disp. 26 , 137–145 (2018).

Tian, P. F. et al. Size-dependent efficiency and efficiency droop of blue InGaN micro-light emitting diodes. Appl. Phys. Lett. 101 , 231110 (2012).

Olivier, F. et al. Shockley-Read-Hall and Auger non-radiative recombination in GaN based LEDs: a size effect study. Appl. Phys. Lett. 111 , 022104 (2017).

Konoplev, S. S., Bulashevich, K. A. & Karpov, S. Y. From large-size to micro-LEDs: scaling trends revealed by modeling. Phys. Status Solidi (A) 215 , 1700508 (2018).

Li, L. Z. et al. Transfer-printed, tandem microscale light-emitting diodes for full-color displays. Proc. Natl Acad. Sci. USA 118 , e2023436118 (2021).

Oh, J. T. et al. Light output performance of red AlGaInP-based light emitting diodes with different chip geometries and structures. Opt. Express 26 , 11194–11200 (2018).

Shen, Y. C. et al. Auger recombination in InGaN measured by photoluminescence. Appl. Phys. Lett. 91 , 141101 (2007).

Wong, M. S. et al. High efficiency of III-nitride micro-light-emitting diodes by sidewall passivation using atomic layer deposition. Opt. Express 26 , 21324–21331 (2018).

Han, S. C. et al. AlGaInP-based Micro-LED array with enhanced optoelectrical properties. Optical Mater. 114 , 110860 (2021).

Wong, M. S. et al. Size-independent peak efficiency of III-nitride micro-light-emitting-diodes using chemical treatment and sidewall passivation. Appl. Phys. Express 12 , 097004 (2019).

Ley, R. T. et al. Revealing the importance of light extraction efficiency in InGaN/GaN microLEDs via chemical treatment and dielectric passivation. Appl. Phys. Lett. 116 , 251104 (2020).

Moon, S. W. et al. Recent progress on ultrathin metalenses for flat optics. iScience 23 , 101877 (2020).

Arbabi, A. et al. Efficient dielectric metasurface collimating lenses for mid-infrared quantum cascade lasers. Opt. Express 23 , 33310–33317 (2015).

Yu, N. F. et al. Light propagation with phase discontinuities: generalized laws of reflection and refraction. Science 334 , 333–337 (2011).

Liang, H. W. et al. High performance metalenses: numerical aperture, aberrations, chromaticity, and trade-offs. Optica 6 , 1461–1470 (2019).

Park, J. S. et al. All-glass, large metalens at visible wavelength using deep-ultraviolet projection lithography. Nano Lett. 19 , 8673–8682 (2019).

Yoon, G. et al. Single-step manufacturing of hierarchical dielectric metalens in the visible. Nat. Commun. 11 , 2268 (2020).

Lee, G. Y. et al. Metasurface eyepiece for augmented reality. Nat. Commun. 9 , 4562 (2018).

Chen, W. T. et al. A broadband achromatic metalens for focusing and imaging in the visible. Nat. Nanotechnol. 13 , 220–226 (2018).

Wang, S. M. et al. A broadband achromatic metalens in the visible. Nat. Nanotechnol. 13 , 227–232 (2018).

Lan, S. F. et al. Metasurfaces for near-eye augmented reality. ACS Photonics 6 , 864–870 (2019).

Fan, Z. B. et al. A broadband achromatic metalens array for integral imaging in the visible. Light.: Sci. Appl. 8 , 67 (2019).

Shi, Z. J., Chen, W. T. & Capasso, F. Wide field-of-view waveguide displays enabled by polarization-dependent metagratings. In Proc. SPIE 10676, Digital Optics for Immersive Displays (SPIE, Strasbourg, France, 2018).

Hong, C. C., Colburn, S. & Majumdar, A. Flat metaform near-eye visor. Appl. Opt. 56 , 8822–8827 (2017).

Bayati, E. et al. Design of achromatic augmented reality visors based on composite metasurfaces. Appl. Opt. 60 , 844–850 (2021).

Nikolov, D. K. et al. Metaform optics: bridging nanophotonics and freeform optics. Sci. Adv. 7 , eabe5112 (2021).

Tamir, T. & Peng, S. T. Analysis and design of grating couplers. Appl. Phys. 14 , 235–254 (1977).

Miller, J. M. et al. Design and fabrication of binary slanted surface-relief gratings for a planar optical interconnection. Appl. Opt. 36 , 5717–5727 (1997).

Levola, T. & Laakkonen, P. Replicated slanted gratings with a high refractive index material for in and outcoupling of light. Opt. Express 15 , 2067–2074 (2007).

Shrestha, S. et al. Broadband achromatic dielectric metalenses. Light.: Sci. Appl. 7 , 85 (2018).

Li, Z. Y. et al. Meta-optics achieves RGB-achromatic focusing for virtual reality. Sci. Adv. 7 , eabe4458 (2021).

Ratcliff, J. et al. ThinVR: heterogeneous microlens arrays for compact, 180 degree FOV VR near-eye displays. IEEE Trans. Vis. Computer Graph. 26 , 1981–1990 (2020).

Wong, T. L. et al. Folded optics with birefringent reflective polarizers. In Proc. SPIE 10335, Digital Optical Technologies 2017 (SPIE, Munich, Germany, 2017).

Li, Y. N. Q. et al. Broadband cholesteric liquid crystal lens for chromatic aberration correction in catadioptric virtual reality optics. Opt. Express 29 , 6011–6020 (2021).

Bang, K. et al. Lenslet VR: thin, flat and wide-FOV virtual reality display using fresnel lens and lenslet array. IEEE Trans. Vis. Computer Graph. 27 , 2545–2554 (2021).

Maimone, A. & Wang, J. R. Holographic optics for thin and lightweight virtual reality. ACM Trans. Graph. 39 , 67 (2020).

Kramida, G. Resolving the vergence-accommodation conflict in head-mounted displays. IEEE Trans. Vis. Computer Graph. 22 , 1912–1931 (2016).

Zhan, T. et al. Multifocal displays: review and prospect. PhotoniX 1 , 10 (2020).

Shimobaba, T., Kakue, T. & Ito, T. Review of fast algorithms and hardware implementations on computer holography. IEEE Trans. Ind. Inform. 12 , 1611–1622 (2016).

Xiao, X. et al. Advances in three-dimensional integral imaging: sensing, display, and applications [Invited]. Appl. Opt. 52 , 546–560 (2013).

Kuiper, S. & Hendriks, B. H. W. Variable-focus liquid lens for miniature cameras. Appl. Phys. Lett. 85 , 1128–1130 (2004).

Liu, S. & Hua, H. Time-multiplexed dual-focal plane head-mounted display with a liquid lens. Opt. Lett. 34 , 1642–1644 (2009).

Wilson, A. & Hua, H. Design and demonstration of a vari-focal optical see-through head-mounted display using freeform Alvarez lenses. Opt. Express 27 , 15627–15637 (2019).

Zhan, T. et al. Pancharatnam-Berry optical elements for head-up and near-eye displays [Invited]. J. Optical Soc. Am. B 36 , D52–D65 (2019).

Oh, C. & Escuti, M. J. Achromatic diffraction from polarization gratings with high efficiency. Opt. Lett. 33 , 2287–2289 (2008).

Zou, J. Y. et al. Broadband wide-view Pancharatnam-Berry phase deflector. Opt. Express 28 , 4921–4927 (2020).

Zhan, T., Lee, Y. H. & Wu, S. T. High-resolution additive light field near-eye display by switchable Pancharatnam–Berry phase lenses. Opt. Express 26 , 4863–4872 (2018).

Tan, G. J. et al. Polarization-multiplexed multiplane display. Opt. Lett. 43 , 5651–5654 (2018).

Lanman, D. R. Display systems research at facebook reality labs (conference presentation). In Proc. SPIE 11310, Optical Architectures for Displays and Sensing in Augmented, Virtual, and Mixed Reality (AR, VR, MR) (SPIE, San Francisco, California, United States, 2020).

Liu, Z. J. et al. A novel BLU-free full-color LED projector using LED on silicon micro-displays. IEEE Photonics Technol. Lett. 25 , 2267–2270 (2013).

Han, H. V. et al. Resonant-enhanced full-color emission of quantum-dot-based micro LED display technology. Opt. Express 23 , 32504–32515 (2015).

Lin, H. Y. et al. Optical cross-talk reduction in a quantum-dot-based full-color micro-light-emitting-diode display by a lithographic-fabricated photoresist mold. Photonics Res. 5 , 411–416 (2017).

Liu, Z. J. et al. Micro-light-emitting diodes with quantum dots in display technology. Light.: Sci. Appl. 9 , 83 (2020).

Kim, H. M. et al. Ten micrometer pixel, quantum dots color conversion layer for high resolution and full color active matrix micro-LED display. J. Soc. Inf. Disp. 27 , 347–353 (2019).

Xuan, T. T. et al. Inkjet-printed quantum dot color conversion films for high-resolution and full-color micro light-emitting diode displays. J. Phys. Chem. Lett. 11 , 5184–5191 (2020).

Chen, S. W. H. et al. Full-color monolithic hybrid quantum dot nanoring micro light-emitting diodes with improved efficiency using atomic layer deposition and nonradiative resonant energy transfer. Photonics Res. 7 , 416–422 (2019).

Krishnan, C. et al. Hybrid photonic crystal light-emitting diode renders 123% color conversion effective quantum yield. Optica 3 , 503–509 (2016).

Kang, J. H. et al. RGB arrays for micro-light-emitting diode applications using nanoporous GaN embedded with quantum dots. ACS Applied Mater. Interfaces 12 , 30890–30895 (2020).

Chen, G. S. et al. Monolithic red/green/blue micro-LEDs with HBR and DBR structures. IEEE Photonics Technol. Lett. 30 , 262–265 (2018).

Hsiang, E. L. et al. Enhancing the efficiency of color conversion micro-LED display with a patterned cholesteric liquid crystal polymer film. Nanomaterials 10 , 2430 (2020).

Kang, C. M. et al. Hybrid full-color inorganic light-emitting diodes integrated on a single wafer using selective area growth and adhesive bonding. ACS Photonics 5 , 4413–4422 (2018).

Geum, D. M. et al. Strategy toward the fabrication of ultrahigh-resolution micro-LED displays by bonding-interface-engineered vertical stacking and surface passivation. Nanoscale 11 , 23139–23148 (2019).

Ra, Y. H. et al. Full-color single nanowire pixels for projection displays. Nano Lett. 16 , 4608–4615 (2016).

Motoyama, Y. et al. High-efficiency OLED microdisplay with microlens array. J. Soc. Inf. Disp. 27 , 354–360 (2019).

Fujii, T. et al. 4032 ppi High-resolution OLED microdisplay. J. Soc. Inf. Disp. 26 , 178–186 (2018).

Hamer, J. et al. High-performance OLED microdisplays made with multi-stack OLED formulations on CMOS backplanes. In Proc. SPIE 11473, Organic and Hybrid Light Emitting Materials and Devices XXIV . Online Only (SPIE, 2020).

Joo, W. J. et al. Metasurface-driven OLED displays beyond 10,000 pixels per inch. Science 370 , 459–463 (2020).

Vettese, D. Liquid crystal on silicon. Nat. Photonics 4 , 752–754 (2010).

Zhang, Z. C., You, Z. & Chu, D. P. Fundamentals of phase-only liquid crystal on silicon (LCOS) devices. Light.: Sci. Appl. 3 , e213 (2014).

Hornbeck, L. J. The DMD TM projection display chip: a MEMS-based technology. MRS Bull. 26 , 325–327 (2001).

Zhang, Q. et al. Polarization recycling method for light-pipe-based optical engine. Appl. Opt. 52 , 8827–8833 (2013).

Hofmann, U., Janes, J. & Quenzer, H. J. High-Q MEMS resonators for laser beam scanning displays. Micromachines 3 , 509–528 (2012).

Holmström, S. T. S., Baran, U. & Urey, H. MEMS laser scanners: a review. J. Microelectromechanical Syst. 23 , 259–275 (2014).

Bao, X. Z. et al. Design and fabrication of AlGaInP-based micro-light-emitting-diode array devices. Opt. Laser Technol. 78 , 34–41 (2016).

Olivier, F. et al. Influence of size-reduction on the performances of GaN-based micro-LEDs for display application. J. Lumin. 191 , 112–116 (2017).

Liu, Y. B. et al. High-brightness InGaN/GaN Micro-LEDs with secondary peak effect for displays. IEEE Electron Device Lett. 41 , 1380–1383 (2020).

Qi, L. H. et al. 848 ppi high-brightness active-matrix micro-LED micro-display using GaN-on-Si epi-wafers towards mass production. Opt. Express 29 , 10580–10591 (2021).

Chen, E. G. & Yu, F. H. Design of an elliptic spot illumination system in LED-based color filter-liquid-crystal-on-silicon pico projectors for mobile embedded projection. Appl. Opt. 51 , 3162–3170 (2012).

Darmon, D., McNeil, J. R. & Handschy, M. A. 70.1: LED-illuminated pico projector architectures. Soc. Inf. Disp. Int. Symp . Dig. Tech. Pap. 39 , 1070–1073 (2008).

Essaian, S. & Khaydarov, J. State of the art of compact green lasers for mobile projectors. Optical Rev. 19 , 400–404 (2012).

Sun, W. S. et al. Compact LED projector design with high uniformity and efficiency. Appl. Opt. 53 , H227–H232 (2014).

Sun, W. S., Chiang, Y. C. & Tsuei, C. H. Optical design for the DLP pocket projector using LED light source. Phys. Procedia 19 , 301–307 (2011).

Chen, S. W. H. et al. High-bandwidth green semipolar (20–21) InGaN/GaN micro light-emitting diodes for visible light communication. ACS Photonics 7 , 2228–2235 (2020).

Yoshida, K. et al. 245 MHz bandwidth organic light-emitting diodes used in a gigabit optical wireless data link. Nat. Commun. 11 , 1171 (2020).

Park, D. W. et al. 53.5: High-speed AMOLED pixel circuit and driving scheme. Soc. Inf. Disp. Int. Symp . Dig. Tech. Pap. 41 , 806–809 (2010).

Tan, L., Huang, H. C. & Kwok, H. S. 78.1: Ultra compact polarization recycling system for white light LED based pico-projection system. Soc. Inf. Disp. Int. Symp. Dig. Tech. Pap. 41 , 1159–1161 (2010).

Maimone, A., Georgiou, A. & Kollin, J. S. Holographic near-eye displays for virtual and augmented reality. ACM Trans. Graph. 36 , 85 (2017).

Pan, J. W. et al. Portable digital micromirror device projector using a prism. Appl. Opt. 46 , 5097–5102 (2007).

Huang, Y. et al. Liquid-crystal-on-silicon for augmented reality displays. Appl. Sci. 8 , 2366 (2018).

Peng, F. L. et al. Analytical equation for the motion picture response time of display devices. J. Appl. Phys. 121 , 023108 (2017).

Pulli, K. 11-2: invited paper: meta 2: immersive optical-see-through augmented reality. Soc. Inf. Disp. Int. Symp . Dig. Tech. Pap. 48 , 132–133 (2017).

Lee, B. & Jo, Y. in Advanced Display Technology: Next Generation Self-Emitting Displays (eds Kang, B., Han, C. W. & Jeong, J. K.) 307–328 (Springer, 2021).

Cheng, D. W. et al. Design of an optical see-through head-mounted display with a low f -number and large field of view using a freeform prism. Appl. Opt. 48 , 2655–2668 (2009).

Zheng, Z. R. et al. Design and fabrication of an off-axis see-through head-mounted display with an x–y polynomial surface. Appl. Opt. 49 , 3661–3668 (2010).

Wei, L. D. et al. Design and fabrication of a compact off-axis see-through head-mounted display using a freeform surface. Opt. Express 26 , 8550–8565 (2018).

Liu, S., Hua, H. & Cheng, D. W. A novel prototype for an optical see-through head-mounted display with addressable focus cues. IEEE Trans. Vis. Computer Graph. 16 , 381–393 (2010).

Hua, H. & Javidi, B. A 3D integral imaging optical see-through head-mounted display. Opt. Express 22 , 13484–13491 (2014).

Song, W. T. et al. Design of a light-field near-eye display using random pinholes. Opt. Express 27 , 23763–23774 (2019).

Wang, X. & Hua, H. Depth-enhanced head-mounted light field displays based on integral imaging. Opt. Lett. 46 , 985–988 (2021).

Huang, H. K. & Hua, H. Generalized methods and strategies for modeling and optimizing the optics of 3D head-mounted light field displays. Opt. Express 27 , 25154–25171 (2019).

Huang, H. K. & Hua, H. High-performance integral-imaging-based light field augmented reality display using freeform optics. Opt. Express 26 , 17578–17590 (2018).

Cheng, D. W. et al. Design and manufacture AR head-mounted displays: a review and outlook. Light.: Adv. Manuf. 2 , 24 (2021).

Google Scholar  

Westheimer, G. The Maxwellian view. Vis. Res. 6 , 669–682 (1966).

Do, H., Kim, Y. M. & Min, S. W. Focus-free head-mounted display based on Maxwellian view using retroreflector film. Appl. Opt. 58 , 2882–2889 (2019).

Park, J. H. & Kim, S. B. Optical see-through holographic near-eye-display with eyebox steering and depth of field control. Opt. Express 26 , 27076–27088 (2018).

Chang, C. L. et al. Toward the next-generation VR/AR optics: a review of holographic near-eye displays from a human-centric perspective. Optica 7 , 1563–1578 (2020).

Hsueh, C. K. & Sawchuk, A. A. Computer-generated double-phase holograms. Appl. Opt. 17 , 3874–3883 (1978).

Chakravarthula, P. et al. Wirtinger holography for near-eye displays. ACM Trans. Graph. 38 , 213 (2019).

Peng, Y. F. et al. Neural holography with camera-in-the-loop training. ACM Trans. Graph. 39 , 185 (2020).

Shi, L. et al. Towards real-time photorealistic 3D holography with deep neural networks. Nature 591 , 234–239 (2021).

Jang, C. et al. Retinal 3D: augmented reality near-eye display via pupil-tracked light field projection on retina. ACM Trans. Graph. 36 , 190 (2017).

Jang, C. et al. Holographic near-eye display with expanded eye-box. ACM Trans. Graph. 37 , 195 (2018).

Kim, S. B. & Park, J. H. Optical see-through Maxwellian near-to-eye display with an enlarged eyebox. Opt. Lett. 43 , 767–770 (2018).

Shrestha, P. K. et al. Accommodation-free head mounted display with comfortable 3D perception and an enlarged eye-box. Research 2019 , 9273723 (2019).

Lin, T. G. et al. Maxwellian near-eye display with an expanded eyebox. Opt. Express 28 , 38616–38625 (2020).

Jo, Y. et al. Eye-box extended retinal projection type near-eye display with multiple independent viewpoints [Invited]. Appl. Opt. 60 , A268–A276 (2021).

Xiong, J. H. et al. Aberration-free pupil steerable Maxwellian display for augmented reality with cholesteric liquid crystal holographic lenses. Opt. Lett. 46 , 1760–1763 (2021).

Viirre, E. et al. Laser safety analysis of a retinal scanning display system. J. Laser Appl. 9 , 253–260 (1997).

Ratnam, K. et al. Retinal image quality in near-eye pupil-steered systems. Opt. Express 27 , 38289–38311 (2019).

Maimone, A. et al. Pinlight displays: wide field of view augmented reality eyeglasses using defocused point light sources. In Proc. ACM SIGGRAPH 2014 Emerging Technologies (ACM, Vancouver, Canada, 2014).

Jeong, J. et al. Holographically printed freeform mirror array for augmented reality near-eye display. IEEE Photonics Technol. Lett. 32 , 991–994 (2020).

Ha, J. & Kim, J. Augmented reality optics system with pin mirror. US Patent 10,989,922 (2021).

Park, S. G. Augmented and mixed reality optical see-through combiners based on plastic optics. Inf. Disp. 37 , 6–11 (2021).

Xiong, J. H. et al. Breaking the field-of-view limit in augmented reality with a scanning waveguide display. OSA Contin. 3 , 2730–2740 (2020).

Levola, T. 7.1: invited paper: novel diffractive optical components for near to eye displays. Soc. Inf. Disp. Int. Symp . Dig. Tech. Pap. 37 , 64–67 (2006).

Laakkonen, P. et al. High efficiency diffractive incouplers for light guides. In Proc. SPIE 6896, Integrated Optics: Devices, Materials, and Technologies XII . (SPIE, San Jose, California, United States, 2008).

Bai, B. F. et al. Optimization of nonbinary slanted surface-relief gratings as high-efficiency broadband couplers for light guides. Appl. Opt. 49 , 5454–5464 (2010).

Äyräs, P., Saarikko, P. & Levola, T. Exit pupil expander with a large field of view based on diffractive optics. J. Soc. Inf. Disp. 17 , 659–664 (2009).

Yoshida, T. et al. A plastic holographic waveguide combiner for light-weight and highly-transparent augmented reality glasses. J. Soc. Inf. Disp. 26 , 280–286 (2018).

Yu, C. et al. Highly efficient waveguide display with space-variant volume holographic gratings. Appl. Opt. 56 , 9390–9397 (2017).

Shi, X. L. et al. Design of a compact waveguide eyeglass with high efficiency by joining freeform surfaces and volume holographic gratings. J. Optical Soc. Am. A 38 , A19–A26 (2021).

Han, J. et al. Portable waveguide display system with a large field of view by integrating freeform elements and volume holograms. Opt. Express 23 , 3534–3549 (2015).

Weng, Y. S. et al. Liquid-crystal-based polarization volume grating applied for full-color waveguide displays. Opt. Lett. 43 , 5773–5776 (2018).

Lee, Y. H. et al. Compact see-through near-eye display with depth adaption. J. Soc. Inf. Disp. 26 , 64–70 (2018).

Tekolste, R. D. & Liu, V. K. Outcoupling grating for augmented reality system. US Patent 10,073,267 (2018).

Grey, D. & Talukdar, S. Exit pupil expanding diffractive optical waveguiding device. US Patent 10,073, 267 (2019).

Yoo, C. et al. Extended-viewing-angle waveguide near-eye display with a polarization-dependent steering combiner. Opt. Lett. 45 , 2870–2873 (2020).

Schowengerdt, B. T., Lin, D. & St. Hilaire, P. Multi-layer diffractive eyepiece with wavelength-selective reflector. US Patent 10,725,223 (2020).

Wang, Q. W. et al. Stray light and tolerance analysis of an ultrathin waveguide display. Appl. Opt. 54 , 8354–8362 (2015).

Wang, Q. W. et al. Design of an ultra-thin, wide-angle, stray-light-free near-eye display with a dual-layer geometrical waveguide. Opt. Express 28 , 35376–35394 (2020).

Frommer, A. Lumus: maximus: large FoV near to eye display for consumer AR glasses. In Proc. SPIE 11764, AVR21 Industry Talks II . Online Only (SPIE, 2021).

Ayres, M. R. et al. Skew mirrors, methods of use, and methods of manufacture. US Patent 10,180,520 (2019).

Utsugi, T. et al. Volume holographic waveguide using multiplex recording for head-mounted display. ITE Trans. Media Technol. Appl. 8 , 238–244 (2020).

Aieta, F. et al. Multiwavelength achromatic metasurfaces by dispersive phase compensation. Science 347 , 1342–1345 (2015).

Arbabi, E. et al. Controlling the sign of chromatic dispersion in diffractive optics with dielectric metasurfaces. Optica 4 , 625–632 (2017).

Download references

Acknowledgements

The authors are indebted to Goertek Electronics for the financial support and Guanjun Tan for helpful discussions.

Author information

Authors and affiliations.

College of Optics and Photonics, University of Central Florida, Orlando, FL, 32816, USA

Jianghao Xiong, En-Lin Hsiang, Ziqian He, Tao Zhan & Shin-Tson Wu

You can also search for this author in PubMed   Google Scholar

Contributions

J.X. conceived the idea and initiated the project. J.X. mainly wrote the manuscript and produced the figures. E.-L.H., Z.H., and T.Z. contributed to parts of the manuscript. S.W. supervised the project and edited the manuscript.

Corresponding author

Correspondence to Shin-Tson Wu .

Ethics declarations

Conflict of interest.

The authors declare no competing interests.

Supplementary information

Supplementary information, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Xiong, J., Hsiang, EL., He, Z. et al. Augmented reality and virtual reality displays: emerging technologies and future perspectives. Light Sci Appl 10 , 216 (2021). https://doi.org/10.1038/s41377-021-00658-8

Download citation

Received : 06 June 2021

Revised : 26 September 2021

Accepted : 04 October 2021

Published : 25 October 2021

DOI : https://doi.org/10.1038/s41377-021-00658-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Color liquid crystal grating based color holographic 3d display system with large viewing angle.

• Qiong-Hua Wang

Light: Science & Applications (2024)

Mass-produced and uniformly luminescent photochromic fibers toward future interactive wearable displays

Effects of virtual reality exposure therapy on state-trait anxiety in individuals with dentophobia.

• Elham Majidi
• Gholamreza Manshaee

Current Psychology (2024)

A review of convolutional neural networks in computer vision

• Milan Parmar

Artificial Intelligence Review (2024)

Visual analytics for digital twins: a conceptual framework and case study

• Hangbin Zheng
• Tianyuan Liu
• Jinsong Bao

Journal of Intelligent Manufacturing (2024)

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Help | Advanced Search

Computer Science > Computer Vision and Pattern Recognition

Title: 3d building reconstruction from monocular remote sensing images with multi-level supervisions.

Abstract: 3D building reconstruction from monocular remote sensing images is an important and challenging research problem that has received increasing attention in recent years, owing to its low cost of data acquisition and availability for large-scale applications. However, existing methods rely on expensive 3D-annotated samples for fully-supervised training, restricting their application to large-scale cross-city scenarios. In this work, we propose MLS-BRN, a multi-level supervised building reconstruction network that can flexibly utilize training samples with different annotation levels to achieve better reconstruction results in an end-to-end manner. To alleviate the demand on full 3D supervision, we design two new modules, Pseudo Building Bbox Calculator and Roof-Offset guided Footprint Extractor, as well as new tasks and training strategies for different types of samples. Experimental results on several public and new datasets demonstrate that our proposed MLS-BRN achieves competitive performance using much fewer 3D-annotated samples, and significantly improves the footprint extraction and 3D reconstruction performance compared with current state-of-the-art. The code and datasets of this work will be released at this https URL .

Submission history

Access paper:.

• HTML (experimental)
• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

• Biology & Environment
• Clean Energy
• Fusion & Fission
• Physical Sciences
• National Security
• Neutron Science
• Supercomputing
• User Facilities
• Educational Programs
• Procurement
• Small Business Programs
• Leadership Team
• Visiting ORNL
• Fact Sheets
• Virtual Tour

Wind Turbine Blade Repurposing and Recycling: Coupling Repurposing Methods with 3D Printing Technology

Fiber-reinforced thermoplastic composites continue to be implemented in several markets because of their strength capacity and light weight. Especially in the wind industry, wind blades are made of this material which makes them ideal for wind energy generation, but blades become a problem at end-of-life. Because recycling thermoplastic composites technologies are still not cost and environmentally effective, research in this space is needed. This paper presents a case study for coupling repurposed wind turbine blades and 3D printing technology. The purpose is to remanufacture decommissioned wind blades into bus stop roof and seating sections to then use recycled glass fiber from a decommissioned wind turbine blade to 3D print structural and/or fixtures needed in the assembly. We provide an overview of the design process, and the development process steps in repurposing and 3D printing the material, delving into key parameters required for a successful implementation.

Researchers

Organizations

3D reflection seismic imaging of natural gas/fluid escape features in the deep-water Orange Basin of South Africa

• Original Research Paper
• Open access
• Published: 25 May 2023
• Volume 44 , article number  17 , ( 2023 )

Cite this article

You have full access to this open access article

• N. G. Maduna 1 ,
• M. S. D. Manzi 1 ,
• J. E. Bourdeau 2 , 3 &
• Z. Jinnah 1  

2400 Accesses

2 Citations

Explore all metrics

Many features indicative of natural gas and oil leakage are delineated in the deep-water Orange Basin offshore South Africa using 3D reflection seismic data. These features are influenced by the translational and compressional domains of an underlying Upper Cretaceous deep-water fold-and-thrust belt (DWFTB) system detaching Turonian shales. The origin of hydrocarbons is postulated to be from both: (a) thermogenic sources stemming from the speculative Turonian and proven Aptian source rocks at depth; and (b) biogenic sources from organic-rich sediments in the Cenozoic attributed to the Benguela Current upwelling system. The late Campanian surface has a dense population of > 950 pockmarks classified into three groups based on their variable shapes and diameter: giant (> 1500 m), crater (~ 700–900 m) and simple (< 500 m) pockmarks. A total of 85 simple pockmarks are observed on the present-day seafloor in the same area as those imaged on the late Campanian surface found together with mass wasting. A major slump scar in the north surrounds a ~ 4200 m long, tectonically controlled mud volcano. The vent of the elongated mud volcano is near-vertical and situated along the axis of a large anticline marking the intersection of the translational and compressional domains. Along the same fold further south, the greatest accumulation of hydrocarbons is indicated by a positive high amplitude anomaly (PHAA) within a late Campanian anticline. Vast economical hydrocarbon reservoirs have yet to be exploited from the deep-water Orange Basin, as evidenced by the widespread occurrence of natural gas/fluid escape features imaged in this study.

Similar content being viewed by others

Structural characteristics and 3D seismic detection of gas migration pathways in the deep‐water Orange Basin, South Africa

V. Mahlalela, M. S. D. Manzi, … R. J. Durrheim

Discovery of pockmarks in the Zengmu Basin, southern South China Sea and the implication

Yanlin Wang, Guanghong Tu, … Jie Liu

Submarine canyon systems focusing sub-surface fluid in the Canterbury Basin, South Island, New Zealand

Priyadarshi Chinmoy Kumar, Tiago M. Alves & Kalachand Sain

Avoid common mistakes on your manuscript.

Introduction

The southwest African passive margin has received much interest pertaining to the economical hydrocarbon systems found within the Orange and more southern Outeniqua basins (e.g., van der Spuy and Sayidini 2022 ). Deep-sea technological advances made by the petroleum industry since the 1970s have allowed the more distal regions of sedimentary basins to be explored. This has promoted interest in the study of complex geological structures known to either host or are indicative of gas/fluid migration. The expression of fluid and gas migration through the sedimentary column is shown by a variety of features such as: gravity collapse structures, mounds, mud volcanoes, surface and buried pockmarks, diapirs, cold seeps and polygonal faulting (Cartwright et al. 2007 ; Hartwig et al. 2012 ; Ho et al. 2012 ). These features are often associated with seismic anomalies such as positive high amplitude anomalies (PHAAs), bottom simulating reflectors (BSRs), and pipe and chimney venting structures (Løseth et al. 2011 ; Gay et al. 2003 ; Cartwright et al. 2007 ; Hustoft et al. 2010 ; Ho et al. 2012 ). Natural gas/fluid escape features identified offshore South Africa and Namibia in the shallow Orange Basin include seafloor and buried pockmarks (Hartwig et al. 2012 ), seismic chimneys (Ben-Avraham et al. 2002 ; Paton et al. 2007 ; Kuhlmann et al. 2010 ; Boyd et al. 2011 ), mud diapers and volcanoes (Ben-Avraham et al. 2002 ; Viola et al. 2005 ), reflecting the basin’s underlying hydrocarbon system. Economical hydrocarbon reserves are proven in the Orange Basin, both in the shallow and deep-water environments (Boyd et al. 2011 ; Isiaka et al. 2017 ; van der Spuy and Sayidini 2022 ). This is reflected by the Namibian Kudu and South African Ibubhesi gas fields which are in production along the shelf, and the recent deep-water hydrocarbon finds in Namibia from the Graaf-1 appraisal and Venus-1X wildcat wells which are directly adjacent to this study (Fig.  1 ). Since the deep-water discoveries in Namibia are in proximity to this study, and the two regions are similar in geology, the potential for proliferous hydrocarbon finds in the distal Orange Basin of South Africa is high.

Map of the SW African margin showing a known and predicted hydrocarbon systems, wells, position of sections in Fig.  3 , and DWFTBs in the South African Orange Basin; and b location of the present study area in green

The deep-water study area sits within a region of well-preserved Upper Cretaceous gravitational collapse structures, referred to as deep-water fold-and-thrust belts (DWFTBs) (Fig.  1 ; de Vera et al. 2010 ; Rowan et al. 2004 ; Nemcok et al. 2005 ; Paton et al. 2007 ). Key elements in the evolution of the Orange Basin have been constrained through various studies mostly using early 2D reflection seismic data, well logs, then much later 3D reflection seismic data (e.g., Light et al. 1993 ; Clemson et al. 1997 ; de Vera et al. 2010 ; Hartwig et al. 2012 ; Dalton et al. 2015 , 2017 ; Collier et al. 2017 ; Baby et al. 2018 ; Mahlalela et al. 2021 ; Maduna et al. 2022 ). These studies describe events within the late stages of continental breakup, including the stratigraphy, structure and formation of gravitational collapse structures together with the hydrocarbon potential of the basin. The observations are however limited in South Africa since few studies have integrated the known hydrocarbon system with the natural fluid/gas features observed in relation to the underlying structure and stratigraphy of the passive margin, and even more so in the distal deep-water environments (e.g., Kuhlman et al. 2010 ; Mahlalela et al. 2021 ). Furthermore, in contrast to the Namibian extent, the deep-water Orange Basin remains underexplored in South Africa due to the sparsity of wells and limited 3D reflection seismic data coverage (Fig.  1 ; PASA 2017 ; van der Spuy and Sayidini 2022 ). The main aim of basin analysis is in understanding the basin’s evolution which is centred around the tectonic setting under which the basin formed, its depositional environment, and the presence, accumulation, and extent of hydrocarbons. In this study we attempt to better constrain these elements in the deep-water Orange Basin offshore South Africa. We use seismic attributes to resolve numerous natural gas/fluid escape features and describe their relationship to the underlying DWFTB system described in Maduna et al. ( 2022 ).

Regional setting

The study area is located in the deep-water Orange Basin offshore SW Africa between 1000 and 2000 m water depths (Fig.  1 ). The basin covers an overall area of ~ 160 000 km 2 with the southern and central extents located offshore western South Africa and the northern extent located offshore Namibia. The Cretaceous to present-day infill of the Orange Basin is derived from the South African Plateau (SAP) (or hinterland) with sediment transported by the Orange and Olifants rivers and their proto-equivalents since the Lower Cretaceous (Maslanyj et al. 1992 ; De Vera et al. 2010 ). The sedimentary succession is comprised of syn-rift sequences related to the break-up of Gondwana during the Late Jurassic, followed by drift (post-rift) sequences with the opening of the South Atlantic (Fig.  2 ; Light et al. 1993 ). In the southern Orange Basin, the maximum depositional thickness is ~ 3 km, while in the north, the sedimentary succession reaches ~ 7 km (Kuhlmann et al. 2010 ). The structural and stratigraphic evolution of the SW African margin has been analysed mostly using seismic data and well logs, including the study of sediment accumulation rates, solid phase volumes and thermo-chronometric datasets (e.g., Light et al. 1993 ; de Vera et al. 2010 ; Hirsch et al. 2010 ; Baby et al. 2018 , 2020 ; Maduna et al. 2022 ).

Chronostratigraphy and sequence stratigraphy of the Orange Basin throughout the evolution of the margin using the stratigraphic nomenclature developed by PetroSA (previously Soeker) (Brown et al. 1995 ; PASA 2017 )

Offshore structural framework

The pre-rift basement offshore SW Africa is comparable to the complex onshore Proterozoic to Early Paleozoic Pan African Gariep Belt (Clemson 1997 ; Frimmel et al. 2011 ; Mohammed et al. 2017 ). A 30 km wide, N–S orientated zone of pronounced flexure, known as the ‘hinge line’, has separated regions of uplift and subsidence since the Mesozoic in the pre-rift basement (Light et al. 1993 ; Clemson et al. 1997 ; Mohammed et al. 2017 ; Baby et al. 2018 ). The hinge line forms a critical boundary separating the margin’s offshore and onshore morphologies (Light et al. 1993 ; Clemson et al. 1997 ; Aizawa et al. 2000 ) and its flexure reflects its progressive landward migration (Baby et al. 2018 ). The hinge line is offset by several NE–SW fracture zones (segment boundaries) which partitioned rifting during the break-up of Gondwana between the Middle to Late Jurassic (160 to 130 Ma) (Clemson et al. 1997 ). Seaward dipping reflectors (SDRs) observed in the upper basement lithologies of the SW African margin reflect the volcanic evolution of the passive margin during rifting (Clemson et al. 1997 ; Menzies et al. 2002 ; Séranne and Anka 2005 ). The SDRs form a thick > 3 km package comparable in age to the Parana-Etendeka Large Igneous Province at ~ 135 Ma (Koopman et al. 2016 ; Baby et al. 2018 ).

The zipper-like south to north opening of the South Atlantic Ocean occurred through right-lateral strike-slip motion along the NE–SW fracture zones (Light et al. 1993 ). Viola et al. ( 2012 ) places the opening of the South Atlantic Ocean at ~ 134 Ma, following ENE-WSW extension and rifting. The spreading oceanic ridge marks the onset of drift; the second major phase of margin evolution following the break-up of Gondwana (Séranne and Anka 2005 ; Granado et al. 2009 ). The spreading centre propagated northwards over a 40 my rift-drift transition period (Viola et al. 2012 ), with drift subdividing the margin into well-defined shelf, slope and basinal environmental settings (Light et al. 1993 ). The oldest evidence of mid-oceanic ridge activity and the transition to oceanic crust occurs at the M3 magnetic anomaly, between the Hauterivian and Barremian sequences at 127 Ma (Séranne and Anka 2005 ). The Orange, Lüderitz and Walvis basins formed in zones of greatest subsidence between the Rio Grande Fracture Zone to the north and Aghulhas-Falkland Fracture Zone to the south (Light et al. 1993 ; Clemson et al. 1997 ; Séranne and Anka 2005 ). The smaller Cape and Outeniqua basins also formed south of the Orange Basin. The predominant structural grain and trend of all structural lineaments is NW–SE to NNW–SSE, following the regional foliation of the SW African margin (Light et al. 1993 ; Wildman et al. 2015 ).

Offshore stratigraphy

The presence of SDRs together with rotated and eroded extensional fault blocks in the syn-rift sequence are evidence of the tectonic control on sedimentation along the SW African margin during this time (Maslanyj et al. 1992 ; Light et al. 1993 ; Granado et al. 2009 ). The syn-rift sequence is comprised of isolated half-grabens infilled with interbedded Late Jurassic to Lower Cretaceous (late Hauterivian) siliciclastic and volcaniclastic sediments (Fig.  2 ; Jungslager 1999 ; Paton et al. 2008 ). The onset of drift began with the deposition of Lower Cretaceous to present-day post-rift clastic sediments above the Hauterivian break-up unconformity (Fig.  2 ; Light et al. 1993 ; Menzies et al. 2002 ; Granado et al. 2009 ; de Vera et al. 2010 ). Early drift sediments correspond to black shales and claystones, while late drift sediments are interbedded heterolithic shales and claystones (Dalton et al. 2017 ).

Sedimentation in the Upper Cretaceous was primarily driven by tectonics with climate and oceanic circulation playing little to no role as the difference in pole and equator temperatures were low (Maslanyj et al. 1992 ; Light et al. 1993 ; Granado et al. 2009 ; Uenzelmann-Neben et al. 2017 ). In the Cenozoic sedimentation and facies distribution were strongly affected by oceanographic processes and climatic changes (Light et al. 1993 ; Weigelt and Uenzelmann-Neben 2004 ). The early Oligocene opening of the Drake Passage in the south created the cooler Antarctic Circumpolar Current and Atlantic meridional overturning circulation currents leading to the subsequent development of thermally stratified bottom and deep-water currents in the Miocene (Weigelt and Uenzelmann-Neben 2004 ; Uenzelmann-Neben et al. 2017 ). Southern-sourced bottom and deep-water currents responsible for depositional changes observed since the Miocene offshore SW Africa include the Antarctic Intermediate Water current, North Atlantic Deep Water, and deep Antarctic Bottom Water currents, together with the Benguela Coastal Current (Weigelt and Uenzelmann-Neben 2004 ). These thermally stratified countercurrents led to cold water upwelling of the Benguela Current in the Miocene intensifying at ~ 11 Ma (Diester-Haass et al. 2004 ; Rommerskirchen et al. 2011 ).

The post-rift stratigraphy of the Orange Basin has been well-described and subdivided in terms of several stratigraphic units/sequences separated by key stratigraphic markers/bounding surfaces (e.g., Emery et al. 1975 ; Bolli et al. 1978 ; Brown et al. 1995 ; Holtar and Forsberg 2000 ; Paton et al. 2008 ; Granado et al. 2009 ; De Vera et al. 2010 ; Kuhlmann et al. 2010 ; Dalton et al. 2017 ; Baby et al. 2018 ; Maduna et al. 2022 ). Several phases of uplift and denudation in the post-rift succession have enhanced gravitational processes within the basin, responsible for both crustal thinning and the inversion of extensional faults (e.g., Granado et al. 2009 ; De Vera et al. 2010 ; Hirsch et al. 2010 ; Brown et al. 2014 ; Wildman et al. 2015 ). Periods of elevated sedimentation rates are recorded mostly in the Upper Cretaceous (93.5–66 Ma), then later in the Oligocene (~ 30–25 Ma) successions (Baby et al. 2020 ). Elevated sedimentary flux in the Upper Cretaceous corresponds to a major uplift of the South African Plateau (De Vera et al. 2010 ; Hirsch et al. 2010 ), caused either by a dynamic topography as the African plate moved over the African Superplume found in the lower mantle, or via lithospheric delamination/metasomatism (Baby et al. 2020 ). Gravitational collapse structures (DWFTB systems) found in the Orange Basin (Fig.  3 ) formed in response to this major margin uplift together with seaward tilting, significantly eroding the inner margin (Paton et al. 2008 ; Granado et al. 2009 ; Hirsch et al. 2010 ; De Vera et al. 2010 ; Kuhlmann et al. 2010 ).

2D seismic profiles through DWFTB systems of the Orange Basin showing; a a full DWFTB system comprised of an up-dip extensional, central transitional (or translational) and down-dip compressional domain (De Vera et al. 2010 ); and b the detailed translational and compressional domains imaged in the present study area (Maduna et al. 2022 ). Abbreviations: Al Albian, Tu Turonian, Sa Santonian, Ma Maastrichtian, e. Ca early Campanian, l. Ca late Campanian, Oli Oligocene, Mio Miocene, and sf seafloor. Vertical exaggeration = 5

Aptian-, Turonian- and Cenomanian-aged maximum flooding surfaces are seaward-dipping, over-pressured shale detachments upon which gravitational sliding of DWFTB systems occur in the Orange Basin (Brown et al. 1995 ; Morley et al. 2011 ; Dalton et al. 2015 ; Baby et al. 2018 ; Maduna et al. 2022 ). A full system consists of a linked: (1) up-dip extensional domain with listric normal faults separating convex-upward growth strata; (2) central transional (or translational) domain with complex overprinted extensional and compressional tectonics; and (3) down-dip compressional domain where folds and thrust faults occur (Fig.  3 ; Rowan et al. 2004 ; Bilotti and Shaw 2005 ; Morley et al. 2011 ). Toe thrusting and folding occurs within an approximately 3 km thick section extending for up to 60 km in the direction of transport in the compressional domain (Morley et al. 2011 ). The study of DWFTB systems has grown because of their association with economic hydrocarbon reserves in the distal folds of anticlines (compressional domain); for example, the northern Orange Basin in Namibia (van der Spuy and Sayidini 2022 ) and the Niger Delta (Bilotti and Shaw 2005 ; Corredor et al. 2005 ; Krueger and Gilbert 2009 ).

Hydrocarbon system

Exploration in the shallow shelf environments of the Orange Basin has confirmed several petroleum systems sourced from Barremian to Albian, and possibly Turonian shales (Aldrich et al. 2003 ; van der Spuy 2003 ). The two commercial hydrocarbon plays in the shallow proximal settings of the South African basin are the Ibhubesi gas field (Fig.  1 ) and A-J oil syn-rift system (van der Spuy and Sayidini 2022 ). The Ibhubesi gas field is sourced from the lower Aptian source shales located in the depocentre of the Orange Basin with reservoirs stratigraphically trapped in fluvial channel-fill sandstones (PASA 2017 ). A similar gas play in the shallow shelf regions is the Kudu gas field, offshore southern Namibia (Fig.  1 ). The Kudu gas field is sourced from Barremian shales with reservoirs stratigraphically trapped within aeolian sandstones (PASA 2017 ). These plays both have the potential of multi-TCF (trillion cubic feet) natural gas reserves. The only oil system found in the shallow shelf region occurs within the isolated A-J half-graben sourced from rich Hauterivian lacustrine shales (Jungslager 1999 ). Oil reservoirs are stratigraphically trapped within lake shoreline sandstones interbedded within the source rocks (PASA 2017 ). Results from the DSDP 361 borehole and a possible Turonian oil source rock imaged the Bredasdorp Basin, indicates that the Orange Basin becomes increasing oil-prone distally (Fig.  1 ; van der Spuy 2003 ; PASA 2017 ). This is proven in Namibia as recent significant light oil discoveries were made from Shell’s Graff-1 appraisal well and TotalEnergies’ Venus-1X wildcat well drilled between 2 and 3 km water depths with reservoirs speculated to be located in the significantly older Aptian to lower Albian sediments (Fig.  1 ; Heins 2022 ; van der Spuy and Sayidini 2022 ). Currently, South Africa’s largest deep-water prospect is found within the southern Outeniqua Basin’s Brulpadda blocks, hosting an average of one billion BOE (barrel of oil equivalent) of natural gas condensate (Feder 2019 ). Due to the harsh conditions imposed by the setting of the strong Agulhas current, however, not much progress has been made in drilling (L’Arvor et al. 2020 ).

Overview of natural gas/fluid escape features

The most well-known seafloor (or palaeo-seafloor) expressions of vertically focussed fluid flow in hydrocarbon systems include mud volcano systems and pockmarks. These surface expressions are linked to subsurface processes such as pipes and chimneys, formed within fracture zones, reflecting the focussed movement of gas, fluids and large sedimentary masses in the case of mud volcanoes (Mazzini and Etiope 2017 ).

Surface expression of natural gas/fluid flow

Mud volcanoes, first recognised and described in the 1970s, are unstable topographical features formed by rapid rates of relative methane expulsion, with rates high enough to remobilize sediment and fluids to the surface (Roberts et al. 2006 ; Judd and Hovland 2007 ; Andresen 2012 ). Mud volcanoes episodically vent a mixture of gas (large amounts of hydrocarbon gas and methane, and lesser amounts of CO 2 , N 2 and He), oil, water, mud and rock fragments (constituents forming the mud breccia) in a process termed ‘sedimentary volcanism’ (Dimitriv 2002 ; Judd and Hovland 2007 ; Mazzini and Etiope 2017 ). Sedimentary volcanism is driven by the gravitative instability of buoyant shales and fluid overpressures, leading to hydrofracturing and the subsequent flow of fluids along permeable fractures (Mazzini and Etiope 2017 ). The most extensive mud volcanoes systems are found in the offshore environment since water-saturated conditions yield low viscosity flows (Mazzini and Etiope 2017 ). In compressional margins Judd and Hovland ( 2007 ) note that there is a relationship between earthquakes and mud volcanoes; a major earthquake may be triggered sedimentary volcanism, which may in turn trigger minor earthquakes. From satellite imagery and field observations, the distribution of mud volcanoes is shown to be often structurally controlled as they occur with normal faults, strike-slip faults, fault-related folds and along the axis of anticlines (Mazzini et al. 2009 ; Mazzini and Etiope 2017 ).

Pockmarks are elliptical or cone-shaped depressions in fine-grained sediment (Hovland and Judd 1988 ), and were first documented by King and MacLean ( 1970 ) on the Scotian shelf. Unlike the large vent feeding a mud volcano, the flux of gas-saturated mud in blowout pipes is moderate, and therefore insufficient to form a large edifice on the seafloor resulting in the formation of smaller pockmarks (Roberts et al. 2006 ; Cartwright 2007 ). The escape of fluids erupting to the surface is envisaged to be quite violent durring the initial formation of pockmarks then followed by smaller seepages along the same migration pathway (Judd and Hovland 2007 ). The size of pockmarks varies depending on the grain size of sediments they are hosted in (Judd and Hovland 2007 ), and type and size of the underlying conduit. Numerous pockmarks have been described both in the shallow (Jungslager 1999 ; Kuhlman et al. 2010 ; Hartwig et al. 2012 ; Isiaka et al. 2017 ; Palan et al. 2020 ) and deep-water reaches (Mahlalela et al. 2021 ) of the Orange Basin on the palaeo-Cenozoic and current seafloors. In the shallow water reaches of the basin, seafloor pockmarks are associated with active faults (e.g., Jungslager 1999 ; Hartwig et al. 2012 ).

Subsurface expression of natural gas/fluid flow

The gas and fluid pathways responsible for the surface expression of gas/fluid escape features are created by discontinuities and unconformities primarily in the form of faults, faulted anticlines, salt diapirs and structural surfaces along the bedrock (Gay et al. 2003 ). A type of faulting system above hydrocarbon reservoirs in fine-grained sediment are polygonal faults describing a honeycombed hydraulic fracturing pattern in planform (Henriet et al. 1991 ; Cartwright 1994 , 2007 ). Polygonal faults are often linked to pockmark formation acting as fluid migration pathways (Cartwright et al. 2003 ; Gay et al. 2006 , 2007 ). Polygonal faults are attributed to mechanisms such as differential compaction and dewatering, overpressures, density inversion and dissolution-induced shear failure (Cartwright et al. 2003 ; Cartwright 2007 , 2011 ).

Seismic chimneys and pipes are the most common subsurface processes reflecting the vertical movement of gas/fluids through fracture systems to the surface (Løseth et al. 2011 ; Gay et al. 2006 , 2007 ; Cartwright et al. 2007 ). Chimneys indicate slow methane expulsion rates, while smaller pipes which lead to surface pockmarks are indicative of moderate methane expulsion rates (Roberts et al. 2006 ). According to Andresen ( 2012 ), a chimney is defined as a wide (sometimes narrow) vertical zone of focussed fluid flow characterised by low amplitude, disrupted or chaotic reflectors; and a pipe is defined as a narrow (< 300 m) vertical zone of focussed fluid flow mainly characterised by stacked, high amplitude anomalous reflectors. There are many hypotheses when it comes to the formation of seismic pipes and chimneys, discussed in detail by Cartwright and Santamarina ( 2015 ). Hypotheses include hydraulic fracturing and erosive fluidization (Brown 1990 ; Løseth et al. 2009 , 2011 ; Cartwright et al. 2007 ), capillary invasion (Cathles et al. 2010 ), localized subsurface volume loss similar to the formation of mud volcano pathways (Roberts et al. 2006 ) and syn-sedimentary formation (Cartwright and Santamarina 2015 ). The most popular hypothesis of pipe and chimney formations is hydraulic fracturing caused by the combined effects of elevated pore fluid pressures with fluid-driven erosion (Cartwright 2007 ). Other subsurface processes include mud intrusions and mounds, domes, dewatering pipes, domes, diapirs and diatremes; all of which are referred to as piercement structures by Mazzini and Etiope ( 2017 ).

Data and methods

Seismic acquisition, processing, and interpretation.

Three deep-water 3D seismic surveys have been conducted in the deep-water extents of the South African Orange Basin (van der Spuy and Sayidini 2022 ). The first acquired in 2002 exhibited low signal-to-noise ratio; then, between 2012 and 2014, two large, higher-resolution 3D seismic surveys were conducted. This present study uses a portion of the northern-most 3D seismic dataset bordering the Namibian maritime licensing region (Fig.  1 ). Shell Global Solutions International commissioned the present study’s high-resolution 3D seismic survey between 2012 and 2013 (Kramer and Heck 2014 ). The survey was conducted in a ~ NNW to SSE orientation, covering a total area of ~ 8200 km 2 (Fig.  1 b). However, in this study, we interpret the northernmost ~ 1800 km 2 portion of the full seismic dataset bordering Namibia.

Seismic acquisition was conducted onboard the Dolphin Geophysical Polar Duchess using the UTM Zone 33S, central meridian 15° map projection. The source used was an array of dual airguns 15 m in length with a separation of 100 m, volume of 4100 cubic inches, 25 m shot point interval (flip/flop) towed at 8 m depths. The group interval and group length for the 7950 m long streamers were both 12.5 m, with each of the 8 streamers being 7950 m in length, separated by 200 m. The data were recorded in SEG-D format with a record length of 7186 ms, sampled at a rate of 2 ms using a low-cut and high-cut frequency of 4.4 Hz at a 12 db/Oct slope and 214 Hz at a 341 db/Oct slope, respectively. Following pre-processing involving data conversion from SEG-D to SEG-Y output onboard the Dolphin Geophysical Polar Duchess , full seismic processing was carried out by the Netherlands Global Processing team using Shell’s proprietary SIPMAP software. Processing was done at 4 ms from SEGY field tape data through surface-related multiple elimination (SRME) using 3D SRME and anisotropic Kirchhoff pre-stack depth migration (PSDM). The original acquisition grid for the inline and crossline was 6.25 m × 50 m, respectively. Following PSDM migration, the final crossline and inline cell size output is given as 25 × 25 m according to Shell’s report for the study area (Kramer and Heck 2014 ). The full acquisition and step-by-step processing parameters are added as tables in the supplementary material.

In this study, the data are geologically interpreted using Schlumberger’s Petrel software. The seismic data has a dominant frequency of 20 Hz and an average velocity of 2400 m/s (see Kuhlman et al. 2010 ). With a wavelength of 120 m, the vertical seismic resolution is calculated to be either 60 or 30 m based on the ½ and ¼ wavelength criteria, respectively (Yilmaz 2001 ). The horizontal resolution for migrated seismic data is defined by ½ the wavelength (i.e., 60 m for these data) (Herron 2011 ). However, this resolution is also dependent on the quality of the data, bin size or trace spacing and geometry of the survey (Lebedeva-Ivanova et al. 2018 ).

Application of seismic attributes

Seismic attributes were applied to the seismic data to image features below the vertical and horizontal resolution limits as in Manzi et al. ( 2013 ) and Sehoole et al. ( 2020 ). Volumetric attributes applied to the full seismic dataset prior to seismic interpretation include variance, generalized spectral decomposition (GSD), envelope, sweetness and the iterative root mean square (RMS) amplitude. Since seismic attributes are sensitive to noise, structural smoothing was first applied to the full seismic volume to pre-condition the data and enhance the signal-to-noise (S/N) ratio (Randen et al. 2000 ). Variance was then applied to the structurally smoothed dataset. Variance is an attribute that highlights discontinuities by measuring local deviations in the seismic signal through a coherency analysis (Silva et al. 2005 ; Maselli et al. 2019 ). Envelope is the most popular trace attribute used to detect hydrocarbons as it highlights acoustic impedance contrasts in sandstone reservoirs as bright spots (Koson et al. 2014 ). The RMS amplitude (iterative) and sweetness attributes are highly correlated to envelope and are therefore also used to detect ‘bright spots’ (i.e., possible hydrocarbon reservoirs). The RMS amplitude (iterative) is a smoother, scaled estimate of the trace envelope (Koson et al. 2014 ). It computes the root mean square (RMS) iteratively on instantaneous trace samples over a user-specified vertical window. The sweetness attribute, defined as envelope divided by the square root of instantaneous frequency, is often used to image coarse-grained sandstone reservoirs. GSD uses the concept of unravelling the seismic waveform back to its pre-computed waveforms and constituent frequencies (Chopra and Marfut 2005 ; Koson et al. 2014 ). Tuning the seismic data to specific frequencies allows subtle changes in lithology or flow barriers to be detected (Chopra and Marfut 2005 ). The GSD attribute is also used as a direct hydrocarbon indicator, known to show gas charged reservoirs (Burnett et al. 2003 ; Naseer et al. 2017 ). Horizon-based attributes were applied once sufficient seismic interpretation had taken place on surfaces of interest. These include the edge detection, and influential data attributes; structural operations applied to the seafloor and late Campanian surfaces to highlight 3D geometric variations introduced by fault displacements and gas/fluid escape features. Edge detection extracts an edge model to enhance discontinuities by combining the dip and dip azimuth properties and normalizing these to the local noise of the surface (Randen et al. 2000 ; Manzi et al. 2012 ). Influential data is essential to ensuring sensible geometric form as it enhances areas of rapid 3D geometric variation. Prior to edge detection and surface smoothing, however, surface smoothing was first applied to filter out anomalous peaks and noise on each surface.

Seismic interpretation strategy

The seismic stratigraphy is described using Mitchum et al. ( 1977 )’s classical approach whereby stratal termination patterns (downlap, onlap, toplap erosional truncation and concordance) separate the sedimentary succession into seismic facies or sequences with distinct internal reflection geometries (e.g., parallel, subparallel, divergent, prograding, chaotic, sigmoid, hummocky). The surfaces that separate these sequences are erosional or conformable stratigraphic markers created through the interaction of sedimentation and sea-level fluctuations as depositional regimes change (Catuneanu 2006 ). The stratigraphic markers used in this study were chosen because of their dominant high amplitudes and lateral continuity observed throughout the seismic volume. Since no wells are present in the deep-water region, a direct well tie could not be performed on the dataset to calibrate the geological ages to the stratigraphy. Geological ages were assigned based on the comparison of previous Orange Basin studies with seismic data and well logs (e.g., Brown et al. 1995 ; de Vera et al. 2010 ; Kuhlmann et al. 2010 ; Hirsch et al. 2010 ; Baby et al. 2018 ). The stratigraphic markers identified in the Cretaceous are the Albian, Turonian, Santonian, early Campanian, late Campanian and Maastrichtian surfaces (Fig.  3 b). According to the offshore stratigraphic nomenclature developed by PetroSA the stratigraphic markers in this study correspond to the 14At1, 15At1, possibly 16Dt, 17At1 and 22At1 unconformities, respectively (Fig.  2 ; Brown et al. 1995 ). In the Cenozoic succession we identified the Miocene and Oligocene erosional unconformities (Fig.  3 b), which were also recognised by Baby et al. ( 2018 ). The sedimentary succession is subdivided into four megasequences (A–D) reflecting the three major phases of margin evolution early drift (A), late drift (B1–C3) and Cenozoic (D1–D3) (Fig.  3 b) as in Dalton et al. ( 2017 ).

Results and interpretations

This study’s seismic volume images the up-dip transitional and down-dip compressional domains of a Upper Cretaceous DWFTB system and the overlying Cenozoic succession between 1000 and 2000 m below sea level (mbsl) (Figs. 1 , 3 b and 4 ). The evidence for fluid and natural gas seepage in the seismic volume includes polygonal faults, pockmarks, a HPAA anticline and a mud volcano, well imaged on the late Campanian surface (Fig.  4 ). The structural framework associated with all these gas/fluid escape features include a combination of thrust faults, normal faults and oblique-slip faults associated with the Upper Cretaceous DWFTB system (Fig.  3 b). Influential data (using two different colour tables) and edge detection were two key horizon-based attributes applied to better enhance all these features, together with the variance volumetric attribute enhancing discontinuities.

3D late Campanian surface covering the full study area (Fig.  1 ) imaged with the influential data attribute (using a white–grey–black colour scale) to highlight features influencing the 3D form of the surface. The surface is highly pockmarked (pockmarks outlined by coloured polygons) with the greatest concentration of pockmarks occurring in the S (bottom left). Other features highlighted include polygonal faults in the E and SE, and an elongated mud volcano with few pockmarks surrounding it in the NW. Vertical exaggeration = 5

Late Campanian pockmarks

There are over 950 well-preserved, elliptical-shaped depressions characteristic of pockmarks on the late Campanian surface (Fig.  4 ). The most efficient horizon-based attribute used to image these pockmarks clearly on the whole surface was influential data using a white–grey–black colour table (see Fig.  4 ). Late Campanian pockmark formation is strongly associated with polygonal faulting in the eastern and southeastern sections of the study area, as many pockmarks of the surface are bound within them (Figs. 4 , 5 and 6 a). The variance time slice placed just above the translational domain of the Upper Cretaceous DWFTB system in Fig.  5 (south eastern portion of Fig.  4 ) displays the honey-combed pattern of polygonal faulting. Faults dip 45°, on average, and have variable dip directions accounting for the polygonal pattern seen in planform. Individual fault throws are 30–50 m and the distance between each fault (the size of each closed polygon cell) ranges from ~ 800 to 1200 m (Fig.  5 ). Most faults in the study area initiate from the Turonian shale detachment surface at depth and terminate around the late Campanian surface (Fig.  3 b). Some of the faults in the eastern section of the study area, however, terminate at the Oligocene or Miocene surfaces (Cenozoic).

3D view of an inline and crossline, together with the variance attribute timeslice (located just blow the late Campanian surface) showing polygonal faults in cross section and their characteristic honey-combed pattern. PHAA positive high amplitude anomaly. Vertical exaggeration = 5

Different types of pockmarks on the late Campanian surface shown using: a the influential data attribute (using a contrast black to white colour table), and b the edge detection attribute on a zoomed in section

Pockmarks on the late Campanian surface were categorized into three different groups based primarily on their size and shape as: giant (8), crater (~ 20) and simple (> 900) pockmarks. Giant pockmarks are greater than 1500 m in diameter with a ~ 1:16 depth-to-diameter ratio (Figs. 6 and 7 ). Crater pockmarks, named so because of their resemblance to simple crater meteorite impact structures, are mostly ~ 800 m in diameter with a 1:7 depth-to-diameter ratio. Crater pockmarks are linked to polygonal faults in the eastern section of the study area (Figs. 4 , 5 and 6 ). Simple pockmarks are regular elliptical-shaped depressions with the smallest sizes (> 500 m diameter and metre-scale deep). All the different types of pockmarks observed on the late Campanian surface are shown in Fig.  6 (a zoomed in portion of Fig.  4 in the SE). Figure  6 a shows the association of different types of pockmarks to surrounding faults and fractures, and Fig.  6 b is a close-up using the edge detection attribute.

Example of a giant pockmark with a smaller, simple pockmark enclosed within it; a inline section showing a chimney leading up to the giant pockmark, b crossline section and interpretation of the giant and simple pockmark, c 3D influential data view of the giant and simple pockmark using the standard red-grey-blue colour scale. Vertical exaggeration = 5

Giant pockmarks are the largest pockmarks identified in the study, being 1500–2000 m in diameter (Figs. 6 and 7 ) and ~ 120 ms TWT (c. 114 m) deep (TWT-depth conversion using velocities of 1900 m/s ± 10%; cf. Kuhlmann et al. 2010 ). There are a few (8) of these large, cone-shaped depressions throughout the study area (e.g., Fig.  6 ) often linked to wide chimneys at depth (Fig.  7 a). An example of a giant pockmark is shown in Fig.  7 . The inline in Fig.  7 a shows a large, dome-shaped anomaly of convex upwards deformed sediments leading up to the feature. This 1800 m wide zone of disrupted reflectors is a seismic chimney. The giant pockmark is bound between normal faults terminating above the Maastrichtian surface. Figure  7 b, a crossline section, shows the pockmark’s direct underlying stratigraphy is disrupted by normal faults. Smaller, simple pockmarks may occur within giant pockmarks as seen in Fig.  7 b and the 3D image in Fig.  7 c.

Pockmarks displaying a “crater-like” morphology in plan and cross-section (Figs. 6 and 8 ) are classified as crater pockmarks in this study. Crater pockmarks are 700–900 m in diameter (Fig.  6 ) and are approximately 120 ms TWT (c. 114 m) deep (TWT-depth conversion using velocities of 1900 m/s ± 10%; cf. Kuhlmann et al. 2010 ). They have slightly raised wing-shaped rims giving them their distinctive “simple crater-like” morphology (Fig.  8 b, c). Compared to giant pockmarks (Fig.  7 ) they are more circular in plan view (Fig.  6 b) and symmetrical in cross-section (Fig.  8 a, b). Crater pockmarks are confined to the SE region of polygonal faulting above the transitional domain (Fig.  6 ). The crater pockmarks are situated directly in the centre of polygonal fault cells (Figs. 6 a and 8 b). Intriguingly, these pockmarks are not associated with large zones of disrupted reflectors directly beneath them but appear to be related to faults and fracture zones (Fig.  8 ).

Example of a crater-type pockmark; a inline section showing two crater pockmarks with one clearly showing a fault leading up to it, b zoomed-in inline section and interpretation of a crater pockmark, c 3D influential data view of the crater pockmark using the standard red–grey–blue colour scale. Vertical exaggeration = 3

Simple pockmarks are much smaller in size, most are metre-scale in depth and less than 300 m in diameter, displaying elliptical- to cone-shaped morphologies (Figs. 7 a, b and 9 a). Furthermore, they lack the outer wing-shaped rims and are distributed throughout the late Campanian surface, unlike crater pockmarks which are confined to polygonal faulted areas above the translational domain (Figs. 4 , 6 and 9 a). Many simple pockmarks occur along faults (Fig.  6 a), and others occur either randomly in between faults or follow a trend in seemingly un-faulted regions indicating possible microfractures (Fig.  9 a). While giant pockmarks are linked to seismic chimneys at depth (Fig.  7 ), both crater and simple pockmarks are linked to faults at depth which are often difficult to distinguish (Figs. 8 and 9 c, d). Pockmarks do not appear to be linked to any pipes at depth (Fig.  8 a), however, this may be due to possible pipe widths being below the seismic resolution limit.

Comparison of the a late Campanian and b seafloor surface’s simple pockmarks using the influential data attribute. In c an inline section is shown cutting through a pair of conjoined pockmarks on the late Campanian surface corresponding to a single pockmark on the seafloor. In d) an arbitrary seismic line following the linear trend of the late Campanian and seafloor pockmarks is imaged, showing the structure between the two surfaces and the underlying DWFTBs in the compressional domain. Abbreviations: Al Albian, Tu Turonian, Sa Santonian, Ma Maastrichtian, e. Ca early Campanian, l. Ca late Campanian, Oli Oligocene, Mio Miocene, and sf seafloor. Vertical exaggeration = 5

Seafloor pockmarks and mass wasting

There are 85 pockmarks imaged on the seafloor with the greatest concentration occurring in the central and southern region of the study in the down-dip direction above compressional domain (Fig.  10 ). Seafloor pockmarks are similar to the late Campanian’s simple pockmarks, although slightly larger in size, with most being ~ 400 m in diameter and less than 120 ms TWT (c. 114 m) deep (TWT-depth conversion using velocities of 1900 m/s ± 10%; cf. Kuhlmann et al. 2010 ). Seafloor pockmarks are strongly influenced by the underlying stratigraphy with most occurring directly above those imaged on the late Campanian surface or following the same trend. An example of this is shown in Fig.  9 a and b; fewer, larger pockmarks occur on the seafloor following the same NE-SW trend of those on the late Campanian surface. A conjoined simple pockmark pair on the late Campanian surface is shown to correspond to one large pockmark on the seafloor in the NW region of Fig.  9 a and b. No other pockmarks are observed in the stratigraphy between the two surfaces, nor are there clearly distinguishable faults, pipes or chimneys leading up from the late Campanian to seafloor surfaces.

Pockmarks and slumping features on the seafloor imaged using a the edge detection, and b influential surface data attributes. The greatest concentration of pockmarks occurs in the S and is often associated with slides (e.g., Fig.  9 ). The NW is dominated by a large slump scar and within the feature an elongated mud volcano outcrops to the surface

The only notable features observed between the late Campanian and seafloor surfaces, following the trend of both surfaces’ pockmarks, are normal faults with some forming the sidewalls (gliding planes) of slides (Fig.  9 c and d). The slides are coherent masses of sediment with little internal deformation. They are ~ 2–4 km in diameter with rotational slip occurring along the Miocene and Oligocene surfaces. In Fig.  9 d original bedding planes are shown to have been rotated by the gliding plane of the slide. Some seafloor pockmarks are found within slides in the centre of the study area in the region directly above a late Campanian PHAA anticline (Fig.), and in the south (Figs. 9 b, d and 10 ). Pockmarks associated with slides are larger in size (up to ~ 1500 m in diameter) and irregular in shape. The greatest mass wasting feature is seen in the NW; a large > 26 × 19 km slump scar characterised by low amplitude, internally deformed reflectors (Figs. 10 , 11 b and 12 a, b). The slump scar is not associated with any pockmarks on the seafloor but with a mud volcano detailed in the following section (Figs. 11 and 12 ).

Mud volcano system. The location of the elongated mud volcano is shown on the late Campanian surface in ( a ) surrounded by simple pockmarks (red solid line = xline position shown in b and c; red stippled line = inline position shown in Fig.  12 ). The cross-section width of the mud volcano is shown in ( b ) situated in the centre of the largest anticline around the intersection of the translational and compressional domains. The interpretation of the mud volcano plumbing system along its width is shown in ( c )

Mud volcano system and slump scar. The cross-section length of an elongated mud volcano is shown in ( a ) situated in the centre of the largest anticline with a major slump scar adjacent to it. The envelope attribute in ( b ) shows acoustically dampened and disrupted reflections within the mud volcano system and the slump deposit compared to their surroundings. The interpretation of the mud volcano plumbing system along its length is shown in ( c )

• Mud volcano

A NW–SE orientated topographical feature is found in the NW of the study area, with a positive relief of 50 ms TWT (c. 48 m) (TWT-depth conversion using velocities of 1900 m/s ± 10%; cf. Kuhlmann et al. 2010 ), a length of ~ 4200 m, and width of ~ 400 m (Figs. 11 and 12 ). This feature is interpreted as an elongated mud volcano. The mud volcano is located within the centre of a large, 7400 m wide anticline from the Upper Cretaceous DWFTB system (Fig.  11 b, c). The anticline marks the intersection between the DWFTB system’s compressional and translational domains. The mud volcano conduit roots from the Turonian surface as a wide ~ 1800 m zone displaying chaotic internal reflections which then narrows into an internally chaotic vent (Figs. 11 b, c and 12 ). Deformation into convex-downwards layers, defining the positive relief of the mud volcano, starts just below the crest of the anticline in Santonian sediments (Fig.  11 c).

On the late Campanian surface, the influential data surface attribute reveals a few simple pockmarks surrounding the mud volcano (Fig.  11 a) which are not as densely populated as those found in the south of the study area (Figs. 4 , 6 and 9 a). The mud volcano is situated within a large, slumped region on the seafloor (Fig.  10 ). Within here, an even deeper depression (slumping within a slump) occurs to the SE, adjacent to the length of the mud volcano characterized by chaotic internal reflections from the Miocene to seafloor surfaces (Fig.  12 a, b). Very few faults surround the mud volcano (Fig.  11 a). Perpendicularly orientated fractures appear to feed into the mud volcano vent using the influential data surface attribute on the late Campanian surface, as presented in Fig.  11 a. Figure  12 is an inline section that cuts through the length of the mud volcano. Reflections within the elongate vent are acoustically dampened and show low amplitudes compared to its surroundings in the envelope attribute (Fig.  12 b).

HPAA anticline

An anticline occurs along the late Campanian surface, appearing as an approximately 3.5 km (Fig.  13 ) by 4.2 km (Fig.  14 ) PHAA. The inline section presented in Fig.  13 shows the anticline is bound by faults dipping away from each other, depicting a horst structure. Four volume-based attributes known to detect the presence of hydrocarbons were used; iterative RMS, sweetness, generalized spectral decomposition (GSD) and envelope (Fig.  13 a–d). The envelope attribute (Fig.  13 d) best illustrated the presence of hydrocarbons with minimal loss in distinguishing strong reflections (surfaces and faults). Figure  14 shows the intersecting crossline section both normally (a) and using the envelope attribute (b) detecting the PHAA of the anticline and the overall interpretation (c). The outer edge from the crest of the anticline is likely the hydrocarbon spoil point which is bound by a fault (Fig.  14 ).

Inline section of an anticline located within a horst structure shown using the: a RMS (iterative), b sweetness, c GSD and d envelope volumetric attributes. Although all volumetric attributes used indicate the presence and accumulation of hydrocarbons along the high amplitude feature, the envelope volumetric attribute in ( d ) highlights the anticline well without distorting the stratigraphy and structure of its surroundings compared to other volumetric attributes

Crossline section showing the stratigraphy and structure of the margin from an Upper Cretaceous DWFTB system underneath younger Cenozoic successions. The uninterpreted seismic section is shown in ( a ). The envelope volumetric attribute shown in ( b ) highlights the greatest accumulation of hydrocarbons as a PHAA defining an anticline, and c is the overall combined interpretation of the inline

Like the previously described mud volcano (Fig.  11 ), the anticline located further S occurs at the intersection of the translational and compressional domains of an underlying Upper Cretaceous DWFTB system (Fig.  14 ). In the region directly above the anticline, large pockmarks and slumps occur on the seafloor, as shown in Fig.  10 . Pockmarks also occur along Campanian surface along the anticline but are difficult to see as they are highly irregular in shape (Fig.  4 ), deviating from the regular elliptical shape. PHAAs are also found within the Turonian to Santonian sediments in the region immediately below the anticline (Figs. 13 and 14 ). Up-dip of the anticline in the NE section of the study, the seismic character of the late Campanian to Maastrichtian sediments becomes progressively higher in amplitude, resulting in another PHAA above the translational domain as shown in Fig.  14 b.

Most faults initiate from the Turonian surface in the compressional domain which often merges with the Albian surface in the translational domain (Fig.  14 c). Thrust faults in the compressional domain terminate at the early Campanian to late Campanian surfaces, while many normal and oblique slip faults from the translational domain terminate along the stratigraphically younger Maastrichtian, Oligocene and Miocene sequence boundaries (Fig.  14 c). Many smaller faults begin within early Campanian to late Campanian sediments in the translational domain and terminate in Cenozoic sediments (Fig.  14 c).

The study of natural gas/fluid flow features is an underutilized tool in basin analysis that may explain various aspects of basin evolution including the timing of formation, main driving and trigger mechanisms, fluid source and migration pathways (Andresen 2012 ). Since the sedimentary succession is undisrupted by salt tectonics, the accumulation and distribution of hydrocarbons in the deep-water Orange Basin can be investigated. Gas/fluid escape features have been classified in several ways according to their geometry, lithology, surrounding geological impact, cause of formation or methane flux intensity (Roberts et al. 2006 ; Cartwright et al. 2007 ; Løseth et al. 2009 ; Huuse et al. 2010 ; Andresen 2012 ). The natural gas/fluid migration features observed in the study area include fluid migration conduits of faults and chimneys, and their surface expression as pockmarks, and a mud volcano on the late Campanian and seafloor surfaces. All these features are strongly controlled by a Upper Cretaceous DWFTB system above or within which they occur (Maduna et al. 2022 ). The processes responsible for forming these gas/fluid escape features are based on the same common principles with one feature forming instead of another due to minor or major differences in influential factors such as the stress environment, sediment and fluid type, fluid concentration, or the influence of external triggers (Judd and Hovland 2007 ).

Upper Cretaceous DWFTB system

Only the up-dip translational and down-dip compressional domains are imaged in the seismic dataset, with gravitational sliding having occurred along a main over-pressurized Turonian shale detachment surface (Fig.  3 b). The compressional domain is characterized by fold and thrust belts recognized throughout the Orange Basin (Figs. 1 and 3 ) along the continental slope in many studies (e.g., Paton et al. 2008 ; de Vera et al. 2010 ; Scarselli et al. 2016 ; Mahlalela et al. 2021 ). The previously ill-defined translational domain is characterized by overprinted extensional (listric normal faults) and compressional (thrust faults) tectonics with the downslope translation of sediment accommodated by extensive oblique-slip faults segmenting thrust sheets along strike (Maduna et al. 2022 ). Known and postulated source rock intervals of the Orange Basin include the Hauterivian, Barremian, Aptian and Turonian shales (Fig.  2 ). The Turonian shale detachment surface is the youngest and most speculative source rock interval estimated to have reached oil and gas maturation between 85 and 16 Ma with temperatures ranging between 100 and 140 °C along the shelf (Hirsch et al. 2010 ). Results from well logs and basin modelling suggest oil maturation in the deep water where sediment is thickest (Aldrich et al. 2003 ; van der Spuy 2003 ; Paton et al. 2008 ; Hirsch et al. 2010 ; van der Spuy and Sayidini 2022 ). The widespread occurrence of subsurface and surface gas/fluid escape features serves as indirect evidence of elevated pore fluid pressures in the Upper Cretaceous. Fluid overpressures are mainly formed by the combined effects of volumetric expansion involved in hydrocarbon generation and maturation, tectonic stresses and disequilibrium compaction (Rowan et al. 2004 ; Bilotti and Shaw 2005 ). Other mechanisms include mechanical compaction (due to sudden mass movement events or gradual burial), dehydration reactions and the thermal effect of increasing temperature gradients in pore fluids (Mazzini and Etiope 2017 ).

The distribution of all subsurface and surface gas/fluid escape features in this study are strongly influenced by the underling tectonics of the Upper Cretaceous DWFTB system. Most of the fluid migration pathways linked to pockmarks on the late Campanian surface originate from the Turonian shale detachment surface (Figs. 7 , 8 , 9 and 12 ). Polygonal faults and crater pockmarks on the late Campanian surface are confined in the SW region above the translational domain (Figs. 4 , 5 and 6 ). Simple pockmarks on the late Campanian and seafloor surfaces are concentrated above the compressional domain (Figs. 4 , 9 and 10 ). At the intersection of the translational and compressional domains, a late Campanian PHAA anticline is found in the center of the study together with irregular seafloor pockmarks in the area directly above it (Figs. 10 , 13 and 14 ). The orientation and location of the elongated mud volcano in the NW also occurs at the intersection of the two domains (Fig.  13 b).

Surface gas/fluid escape features

According to Judd and Hovland ( 2007 ), the density of pockmarks is dependent on the thickness, strength and permeability of the surrounding sediments. The overall distribution of pockmarks in the N of the study area are not as dense as those in the S for both the late Campanian and seafloor surfaces (Figs. 4 and 10 ). This implies low permeabilities, meaning fewer migration pathways resulting in the sparse distribution of pockmarks in the north compared to the south of the study area. Pockmarks in the deep-water study have been influenced by faults and chimneys leading up to the late Campanian and seafloor surfaces. On the late Campanian surface, wide chimneys are linked to giant pockmarks (Fig.  7 a), and faults are linked to simple and crater pockmarks (Figs. 8 a and 9 c, d). All pockmarks are associated with bright(er) spots within the already high amplitude late Campanian and seafloor surfaces they are found along. Although the cross sections in Fig.  9 c and d do not show a distinct association between the two pockmarked surfaces, the fact that seafloor pockmarks occur in the same area or follow the same trend as those of the late Campanian mean that there is a relationship. A possible explanation is that a fracture system is at play, with displacements being below the smallest vertical resolution limit of 30 m between the two surfaces and below the late Campanian. A few giant pockmarks have also identified in the Barents Sea associated with melting ice, and three in the North Sea with one being the approximate size of a football field (900 m wide and 450 m deep) (Judd et al. 1994 ; Judd and Hovland 2007 ). Judd and Hovland ( 2007 ) attribute the formation of these giant pockmarks to the process of explosive decompression, whereby high fluid overpressures penetrate the surface in extreme cases such as the elongated Lokbatan-type mud volcanoes. Since giant pockmarks indicate rapid rates of methane expulsion (Roberts et al. 2006 ; Judd and Hovland 2007 ), so are their connected subsurface chimneys (Fig.  7 ).

Crater pockmarks are confined to the SE, located centrally within cells of polygonal faults (Figs. 4 and 6 ). Polygonal faults are found between the Santonian to Miocene sediments, have variable strike orientations, small fault throws and lateral extensions, and high densities (Figs. 4 , 5 and 6 ). The variable strike orientations indicate that they do not have a principal stress direction (Fig.  5 ) and were thus not formed by regional compressional or extensional tectonics, but rather hydraulic fracturing (Cartwright 2007 ). Andresen and Huuse ( 2011 ) informally termed pockmarks associated with polygonal faulting as ‘bulls-eye’ pockmarks because of their central location within the faults. These ‘bulls-eye’ pockmarks are found within Plio-Pleistocene sediments of the Lower Congo Basin and have large size ranges, measuring between 70 and 500 m in diameter and 20–50 m in depth. In contrast to the ones observed on the Late Campanian surface in this study, those of the Congo Basin occur as a stacked succession in the stratigraphy. The concentric arrangement of pockmarks within, rather than above, polygonal faults strongly suggests that crater pockmark formation predates polygonal faulting (Andresen and Huuse 2011 ).

On the late Campanian surface, simple pockmarks dominate in the SW to NW compressional domain region, occurring along faults, in linear belts unrelated to visible faults and conjoined composite clusters (Fig.  9 a), within giant pockmarks (Figs. 6 , 7 b and 11 a), and otherwise random distributions (Fig.  4 ). The few pockmarks observed on the seafloor have a strong spatial relationship to those on the late Campanian surface with most displaying a simple circular morphology in plan view (Figs. 9 a, b and 10 ). Directly south of the present study, Mahlalela et al. ( 2021 ) describe a few simple pockmarks on the seafloor that are 700–1100 m in diameter with depressions 75–103 m deep. These pockmarks represent the same pockmark field observed on the seafloor surface in this study.

The morphology of mud volcanoes is controlled by several dynamic and mechanical factors, including: the frequency and vigour of volcanism, width of the conduit, pre-existing local topography, erosion type (e.g., bottom currents, wind, rain), rate of basin subsidence and the thickness and character of the affected strata (Mazzini and Etiope 2017 ). Cone-shaped mud volcanoes, the most common morphology, are observed in the shallow reaches of the Orange Basin following a N/NNW trend which is strongly associated with active, near-vertical strike-slip faults, thus reflecting neo-tectonic activity (Ben-Avraham et al. 2002 ; Viola et al. 2005 ). According to Viola et al. ( 2005 ) the mud volcanoes are the offshore structural expression of the same stress field observed onshore Namaqualand, South Africa and central Namibia, where recent faulting created N/NNW and NW orientated lineaments. Both offshore and onshore structures follow the present-day stress field of SW Africa and are thus postulated to be attributed to the Wegener stress anomaly (since the greatest horizontal compressive stress is NW/NNW). The mud volcano observed in this deep-water study also trends NW, however, unlike those observed in the proximal Orange Basin, it is elongated in morphology. An onshore example of an elongated mud volcano is the Lokbatan located in Azerbaijan (Planke et al. 2003 ; Mazzini and Etiope 2017 ). The Lokbatan has provided the basis of what is known of elongated mud volcanoes worldwide since it is easily accessible. This elongated mud volcano is characterised by extremely explosive eruptions with the last recorded in 2001 ejecting mud breccia after a large initial burst of hot methane (Planke et al. 2003 ; Mazzini and Etiope 2017 ). The volume of the Lokbatan mud volcano is affected by the deflation of the underlying shallow chamber following each eruption (Planke et al. 2003 ).

In like manner as the Lokbatan mud volcano coinciding with the trend of an anticline axis (Mazzini and Etiope 2017 ), the elongated mud volcano in this study is also situated along the axis of a deep-seated anticline (Fig.  11 b, c) and is therefore tectonically controlled. The anticline forms part of a Upper Cretaceous fold and thrust belt marking the intersection of the translational and down-dip compressional domains. Sedimentary volcanism is postulated to have begun in the Santonian (86–83 Ma) since deformation shown by convex pull-up reflectors starts within these sediments (Figs. 11 and 12 ). This age coincides with the start of the oil and gas maturation window at 85 Ma estimated by Hirsch et al. ( 2010 ) for the Turonian shale detachment surface from which the mud volcano roots from (Figs. 11 and 12 ). The mud volcano gradually increased in height shown by stratigraphic surfaces leading up to the seafloor which could possibly indicate intense eruptions.

Source of hydrocarbons

Since pockmarks and mud volcanoes are found in a wide variety of settings, from passive to active margin settings, in compressional zones such as accretionary prisms, fold and thrust belt systems, deltaic settings and deep sedimentary basins related to active plate margins (e.g., Judd and Hovland 1988 ; Gay et al. 2003 , 2006 , 2007 ; Loncke et al. 2004 ; Hustoft et al. 2010 ; Andresen and Huuse 2011 ; Ho et al. 2012 ; Hartwig et al. 2012 ; Anka et al. 2014 ), the source of their hydrocarbon system varies greatly. From the synthesis of past studies in the shallow Orange Basin (Jungslager 1999 ; Ben-Avraham et al. 2002 ; Kuhlmann et al. 2010 ; Hartwig et al. 2012 ), we postulate the source of hydrocarbons in this study to also be of both thermogenic and biogenic (or microbial) origin. According to van der Spuy ( 2003 ) and Paton et al. ( 2007 ), thermogenic gas originates from the deep, thermally mature Aptian source shales which become progressively oil-prone distally (Jungslager 1999 ). Since the Turonian shale detachment surface is also a speculated source rock interval, thermogenic hydrocarbons may have been sourced from both the deep-seated Aptian (unobserved in this study) and Turonian shales in this study. Fluid migration pathways stemming from the Turonian surface, together with late Campanian pockmarks linked to them, may therefore consist of thermogenic hydrocarbons. Biogenic gas is believed to originate from younger, organic-rich sediments involved in the upwelling of the Benguela Current in the Cenozoic (Kuhlmann et al. 2010 ).

An increase in biogenic activity attributed to the Benguela current upwelling system is recorded in the late Miocene to early Pliocene sediments offshore SW Africa (Diester-Haass et al. 2004 ; Rommerskirchen et al. 2011 ). This period of upwelling coincides with the development of thermally stratified bottom and deep-water currents offshore SW Africa resulting in major depositional changes observed in the Miocene (Weigelt and Uenzelmann-Neben 2004 ; Maduna et al. 2022 ). Seafloor pockmarks in this study are postulated to consist of a mixture of both thermogenic and biogenic hydrocarbons since the faults feeding them initiate from within the Cenozoic sediments; at the Oligocene and Miocene surfaces (Fig.  9 c, d). The dissociation of gas hydrates is another possible origin for the source of hydrocarbons. Ben-Avraham et al. ( 2002 ) related their mud volcanoes in the shallower extents of the Orange Basin to a gas hydrate stability zone (GHSTZ) at depth. There is no evidence to support this hypothesis in the present deep-water study since no bottom-simulating reflectors (BSR) or other definitive gas hydrate indicators are observed in the seismic volume.

Migration and accumulation of hydrocarbons

The distribution of surface and subsurface gas/fluid escape features shows the overall intensification of methane flux increases from the SE to the NW. In the SE, crater pockmarks on the late Campanian surface are associated with faults (moderate methane flux) (Figs. 4 and 6 ). In the NW, an elongated mud volcano (high methane flux) is found surrounded by a large slump scar on the seafloor (Figs. 4 and 10 ). Unlike the heavily faulted E, SE and central areas of the study, the NW has very few faults (Fig.  4 ). Consequently, instead of fluid flow pressures being distributed along many faults, one near-vertical fault-turned-vent took all the pressure, erupting gas, fluids and sediment to the palaeo- and current seafloors through an elongated mud volcano. Gas and fluids are postulated to have been escaping to the ocean/atmosphere since the Santonian when sedimentary volcanism began which may account for the lack of PHAAs within or surrounding the mud volcano (Figs. 11 b and 12 b). PHAAs are mostly found in the central and SW regions of the study (Figs. 5 , 13 and 14 ). In addition to migrating upwards along fault surfaces, over-pressurized fluids may also travel through the permeable beds (e.g., Gay et al. 2006 ). This is seen in the centre of the study area where an anticline on the late Campanian surface displays a PHAA using the envelope attribute in Fig.  14 b. The PHAA anticline is situated directly above the intersection of the compressional and translational domains and is bound within a horst structure (Figs. 13 and 14 ). Hydrocarbons are therefore both structurally (anticline and horst structure) and stratigraphically (late Campanian surface) trapped. As sedimentation progressed, hydrocarbons migrated upwards through the stratigraphy to the seafloor when later faults formed. Irregular pockmarks on the seafloor, directly above the PHAA anticline, may be explained by this continued seep of hydrocarbons disrupting their normal elliptical/cone-shaped depressions (Figs. 10 and 14 ).

Mass wasting features

Slides are observed in the S and centre of the study area, and a major slump scar is observed in the NW on the present-day seafloor (Fig.  10 ). Slides are known to be coherent masses with minor internal deformation as shown in this study (Fig.  9 d), while slumps (a mass flow that is a magnitude higher in flow velocity) are characterised by more internal deformation as seen from the seafloor sediments in Figs. 11 b and 12 (Posamentier and Martinsen 2011 ). These mass wasting features are closely related to gas/fluid escape features found on the seafloor; an elongated mud volcano is found in the centre of the slump scar in the NW (Figs. 10 , 11 and 12 ) and pockmarks are found together with slides in the S (Figs. 9 b, d and 10 ). Both mass wasting features occur in the distal down-dip region where the slope dips steepest. Possible triggers for mass wasting on the seafloor and the underlying Upper Cretaceous DWFTB system (a different type of mass transport) include seismicity, rapid sedimentation rates upon a dipping slope, high internal pore fluid pressures and downslope undercutting (Séranne and Anka 2005 ; Rogers and Rau 2006 ; Kuhlmann et al. 2010 ). Another trigger mechanism for mass wasting in the Cenozoic is the interaction of opposite-flowing ocean currents along the SW African margin (Weigelt and Uenzelmann-Neben 2004 ).

High methane expulsion rates associated with sedimentary volcanism (Roberts et al. 2006 ) destabilized sediments surrounding the elongated mud volcano. This led to major slumping on the seafloor. The size of the slump scar is an indication of how much sediment was (and possibly still is) being remobilized in the underlying stratigraphy- greater than 26 × 19 km. Pockmarks formed under moderate hydrocarbon expulsion rates (Roberts et al. 2006 ) and consequently, instead of one large slump feature, they are associated with smaller slides which reflect lower flow velocities (Fig.  10 ). Slope instability leading to these mass wasting features along a dipping slope arise from discontinuities in the subsurface. These discontinuities are faults between the Oligocene to seafloor surfaces leading to pockmark formation and are the gliding planes of some slumps (Figs. 9 c and d). Large and irregular seafloor pockmarks in association with slides (Fig.  10 ) reflect the progressive downslope flow of sediment in mass wasting, i.e., pockmarks grow in size as slides continue downslope along a gliding plane as seen in Fig.  9 d. On the seafloor this implies that Cenozoic faults first led to pockmark formation, and some of these very same faults became gliding planes for mass sediment flows (slides). Cenozoic mass wasting features are also observed in the shallower regions of the Orange Basin caused by margin instability and gravity faulting as old, underlying faults were rejuvenated in the outer margin (e.g., Hirsch et al. 2010 ; Palan et al. 2020 ).

Conclusions

The 3D seismic data helped to view natural gas and fluid escape features which would previously have been missed or not fully resolved in a regular 2D survey. This study describes the occurrence of widespread natural gas/fluid escape features in relation to an underlying Upper Cretaceous DWFTB system in the deep-water Orange Basin. Together with the shallow water hydrocarbon systems in play, and encouraging deep-water discoveries in Namibia, the numerous gas/fluid escape features observed in this study point to a proliferous hydrocarbon system in the deep-water South African basin that is yet to be exploited. The following conclusions can be drawn with regards to the origin, distribution and occurrence of gas/fluid escape features, and its implications on the deep-water basin’s hydrocarbon system:

Hydrocarbons are biogenic and mostly thermogenic and in origin, sourced from nutrient-rich Cenozoic sediments and the speculated Turonian source rock, respectively, in addition to the thermogenic Aptian source shales.

Methane expulsion rates increase from the SE to NW having culminated in an elongated mud volcano situated within an anticline axis of a Upper Cretaceous thrust belt marking the intersection of the translational and compressional domains. The tectonically controlled mud volcano is indicative of extreme overpressures, and thus, rapid rates of methane flux.

On the seafloor, sediment instability resulted in major slumping in the NW due to sediment remobilization associated with mud volcanism, and smaller slides in the S associated with pockmarks.

Hydrocarbons migrated in a NW direction following the underlying trend of a Upper Cretaceous thrust belt located at the intersection of the translational and compressional domains.

The largest accumulation of hydrocarbons, and hence most favourable place to drill is in the region directly above the late Campanian anticline based on it appearing as a PHAA, and the presence of seafloor pockmarks directly above the area.

We recommend drilling to ground truth the observations made to get the exact stratigraphy, ages, depths, rates of gas expulsion and nature of the sediments. This will ultimately determine the economic viability of the deep-water Orange Basin’s hydrocarbon system.

Data availability

The data are the property of Shell and may be purchased through the Petroleum Agency of South Africa’s (PASA) online geoportal at https://geoportal.petroleumagencysa.com/Storefront/Viewer/index_map.html (last access: 9 November 2022).

Aizawa M, Bluck B, Cartwright J, Milner S, Swart R, Ward J (2000) Constraints on the geomorphological evolution of Namibia from the offshore stratigraphic record. Commun Geol Surv Namibia 12:337–346

Google Scholar  

Aldrich, J, Zilinski, R, Edman, J, Leu, W, Berge, T and Corbett, K (2003) Documentation of a new petroleum system in the South Atlantic. In AAPG Annual Convention Salt Lake City, p. 90013

Andresen KJ (2012) Fluid flow features in hydrocarbon plumbing systems: What do they tell us about the basin evolution? Mar Geol 332:89–108. https://doi.org/10.1016/j.margeo.2012.07.006

Article   Google Scholar  

Andresen KJ, Huuse M (2011) ‘Bulls-eye’pockmarks and polygonal faulting in the Lower Congo Basin: relative timing and implications for fluid expulsion during shallow burial. Mar Geol 279:111–127. https://doi.org/10.1016/j.margeo.2010.10.016

Anka Z, Loegering M, di Primio R, Marchal D, Rodriguez J, Vallejo E (2014) Distribution and origin of natural gas leakage in the Colorado Basin, offshore Argentina Margin, South America: seismic interpretation and 3D basin modelling. Geol Acta 12:269–285. https://doi.org/10.1344/GeologicaActa2014.12.4.1

Baby G, Guillocheau F, Morin J, Ressouche J, Robin C, Broucke O, Dall’Asta M (2018) Post-rift stratigraphic evolution of the Atlantic margin of Namibia and South Africa: implications for the vertical movements of the margin and the uplift history of the South African Plateau. Mar Petrol Geol 97:169–191. https://doi.org/10.1016/j.marpetgeo.2018.06.030

Baby G, Guillocheau F, Braun J, Robin C, Dall’Asta M (2020) Solid sedimentation rates history of the Southern African continental margins: implications for the uplift history of the South African Plateau. Terra Nova 32:53–65. https://doi.org/10.1111/ter.12435

Ben-Avraham Z, Smith G, Reshef M, Jungslager E (2002) Gas hydrate and mud volcanoes on the southwest African continental margin off South Africa. Geology 30:927–930. https://doi.org/10.1130/0091-7613(2002)030%3c0927:GHAMVO%3e2.0.CO;2

Bilotti F, Shaw JH (2005) Deep-water Niger Delta fold and thrust belt modeled as a critical-taper wedge: the influence of elevated basal fluid pressure on structural styles. AAPG Bull 89:1475–1491. https://doi.org/10.1306/06130505002

Bolli HM, Ryan WBF, Foresman JB, Hottman WE, Kagami H, Longoria JF, McKnight BK, Melguen M, Natland J, Proto-Decima F (1978) Cape basin continental rise sites 360 and 361. Init Rep Deep Sea Drill Proj 40:29–182

Boyd D, Anka Z, di Primio R, Kuhlmann G, De Wit MJ (2011) Passive margin evolution and controls on natural gas leakage in the Orange Basin, South Africa. South Afr J Geol 114:415–432. https://doi.org/10.2113/gssajg.114.3-4.415

Brown KM (1990) The nature and hydrogeological significance of mud diapirs and diatremes for accretionary systems. J Geophys Res 95:8969–8982. https://doi.org/10.1029/JB095iB06p08969

Brown R, Summerfield M, Gleadow A, Gallagher K, Carter A, Beucher R, Wildman M (2014) Intracontinental deformation in southern Africa during the late cretaceous. J Afr Earth Sci 100:20–41

Brown LF, Benson JM, Brink GJ, Doherty S, Jollands A, Jungslager EAH, Keenan JHG, Muntingh A, van Wyk NJS (1995) Sequence stratigraphy in offshore south african divergent basins: an atlas on exploration for cretaceous lowstand traps by soekor. AAPG Studies in Geology, Tulsa, p 191

Burnett MD, Castagna JP, Méndez-Hernández E, Rodríguez GZ, García LF, Vázquez JTM, Avilés MT, Villaseñor RV (2003) Application of spectral decomposition to gas basins in Mexico. Lead Edge 22:1130–1134. https://doi.org/10.1190/1.1634918

Cartwright JA (1994) Episodic basin-wide fluid expulsion from geopressured shale sequences in the North Sea basin. Geology 22:447–450. https://doi.org/10.1130/0091-7613(1994)022%3c0447:EBWFEF%3e2.3.CO;2

Cartwright J (2007) The impact of 3D seismic data on the understanding of compaction, fluid flow and diagenesis in sedimentary basins. J Geol Soc Lond 164:881–893. https://doi.org/10.1144/0016-76492006-14

Cartwright J (2011) Diagenetically induced shear failure of fine-grained sediments and the development of polygonal fault systems. Mar Petrol Geol 28:1593–1610. https://doi.org/10.1016/j.marpetgeo.2011.06.004

Cartwright J, Santamarina C (2015) Seismic characteristics of fluid escape pipes in sedimentary basins: implications for pipe genesis. Mar Petrol Geol 65:126–140. https://doi.org/10.1016/j.marpetgeo.2015.03.023

Cartwright J, Huuse M, Aplin A (2007) Seal bypass systems. AAPG Bull 91:1141–1166. https://doi.org/10.1306/04090705181

Cartwright J, James D, Bolton A (2003) The genesis of polygonal fault systems: a review. Subsurface sediment mobilization. Geol Soc Lond Spec Publ 216:223–243. https://doi.org/10.1144/GSL.SP.2003.216.01.15

Cathles LM, Su Z, Chen D (2010) The physics of gas chimney and pockmark formation, with implications for assessment of seafloor hazards and gas sequestration. Mar Petrol Geol 27:82–91. https://doi.org/10.1016/j.marpetgeo.2009.09.010

Catuneanu O (2006) Principles of sequence stratigraphy. Elsevier, Amsterdam, p 375

Chopra S, Marfurt KJ (2005) Seismic attributes — A historical perspective. Geophysics 70:3SO-28SO. https://doi.org/10.1190/1.2098670

Clemson J, Cartwright J, Booth J (1997) Structural segmentation and the influence of basement structure on the Namibian passive margin. J Geol Soc 154:477–482. https://doi.org/10.1144/gsjgs.154.3.0477

Collier JS, McDermott C, Warner G, Gyori N, Schnabel M, McDermott K, Horn BW (2017) New constraints on the age and style of continental breakup in the South Atlantic from magnetic anomaly data. Earth Planet Sc Lett 477:27–40. https://doi.org/10.1016/j.epsl.2017.08.007

Corredor F, Shaw JH, Bilotti F (2005) Structural styles in the deep-water fold and thrust belts of the Niger Delta. AAPG Bull 89:753–780. https://doi.org/10.1306/02170504074

Dalton TJS, Paton DA, Needham T, Hodgson N (2015) Temporal and spatial evolution of deepwater fold thrust belts: Implications for quantifying strain imbalance. Interpretation 3:SAA59–SAA70. https://doi.org/10.1190/INT-2015-0034.1

Dalton TJS, Paton DA, Needham DT (2017) Influence of mechanical stratigraphy on multi-layer gravity collapse structures: insights from the Orange Basin, South Africa. Geol Soc Lond Sp Publ 438:211–228. https://doi.org/10.1144/SP4384

de Vera J, Granado P, McClay K (2010) Structural evolution of the Orange Basin gravity-driven system, offshore Namibia. Mar Petrol Geol 27:223–237. https://doi.org/10.1016/j.marpetgeo.2009.02.003

Diester-Haass L, Meyers PA, Bickert T (2004) Carbonate crash and biogenic bloom in the late Miocene: evidence from ODP Sites 1085, 1086, and 1087 in the Cape Basin, southeast Atlantic Ocean. Paleoceanography. https://doi.org/10.1029/2003PA000933

Dimitrov LI (2002) Mud volcanoes—the most important pathway for degassing deeply buried sediments. Earth Sci Rev 59:49–76. https://doi.org/10.1016/S0012-8252(02)00069-7

Emery KO, Uchupi E, Bowin CO, Phillips J, Simpson ESW (1975) Continental margin off Western Africa: Cape St Francis (South Africa) to Walvis Ridge (South-West Africa). AAPG Bull 59:3–59. https://doi.org/10.1306/83D91C09-16C7-11D7-8645000102C1865D

Feder J (2019) Offshore: making a comeback after the downturn. J Pet Technol 71:27–31. https://doi.org/10.2118/0519-0027-JPT

Frimmel HE, Basei MS, Gaucher C (2011) Neoproterozoic geodynamic evolution of SW-Gondwana: a southern African perspective. Int j Earth Sci Geol 100:323–354. https://doi.org/10.1007/s00531-010-0571-9

Gay A, Lopez M, Cochonat P, Sultan N, Cauquil E, Brigaud F (2003) Sinuous pockmark belt as indicator of a shallow buried turbiditic channel on the lower slope of the Congo Basin, West African Margin. Geol Soc Lond Spec Publ 216:173–189. https://doi.org/10.1144/GSL.SP.2003.216.01.12

Gay A, Lopez M, Cochonat P, Séranne M, Levaché D, Sermondadaz G (2006) Isolated seafloor pockmarks linked to BSRs, fluid chimneys, polygonal faults and stacked Oligocene-Miocene turbiditic palaeochannels in the Lower Congo Basin. Mar Geol 226:25–40. https://doi.org/10.1016/j.margeo.2005.09.018

Gay A, Lopez M, Berndt C, Seranne M (2007) Geological controls on focused fluid flow associated with seafloor seeps in the Lower Congo Basin. Mar Geol 244:68–92. https://doi.org/10.1016/j.margeo.2007.06.003

Granado P, Vera JD, McClay KR (2009) Tectonostratigraphic evolution of the Orange Basin, SW Africa. Trabajos de geología 29:321–328

Hartwig A, Anka Z, di Primio R (2012) Evidence of a widespread paleo-pockmarked field in the Orange Basin: an indication of an early Eocene massive fluid escape event offshore South Africa. Mar Geol 332–334:222–234. https://doi.org/10.1016/j.margeo.2012.07.012

Heins WA (2022) Probing the frontier in Namibia. First Break 40:51–57. https://doi.org/10.3997/1365-2397.fb2022037

Henriet JP, De Batist M, Verschuren J (1991) Early fracturing of Palaeogene clays, southernmost North Sea: relevance to mechanisms of primary hydrocarbon migration. Spec Pub Eur Assoc Petrol 1:217–227

Herron DA (2011) Resolution. In: Latimer RB (ed) First steps in seismic interpretation. Soc Expl Geophys, Tulsa, pp 75–82

Chapter   Google Scholar  

Hirsch KK, Scheck-Wenderoth M, van Wees J-D, Kuhlmann G, Paton DA (2010) Tectonic subsidence history and thermal evolution of the Orange Basin. Mar Petrol Geol 2:565–584. https://doi.org/10.1016/j.marpetgeo.2009.06.009

Ho S, Cartwright JA, Imbert P (2012) Vertical evolution of fluid venting structures in relation to gas flux, in the Neogene-Quaternary of the Lower Congo Basin, Offshore Angola. Mar Geol 332:40–55. https://doi.org/10.1016/j.margeo.2012.08.011

Holtar E, Forsberg AW (2000) Postrift development of the Walvis Basin, Namibia: results for the exploration campaign in quadrant 1911. In: Mello MR, Katz BJ (eds) Petroleum Systems of South Atlantic Margins. AAPG Mem, Tulsa, pp 429–446

Hovland M, Judd AG (1988) Seabed pockmarks and seepages: impact on geology, biology and the marine environment. Graham and Trotman Ltd, London, p 293

Hustoft S, Bünz S, Mienert J (2010) Three-dimensional seismic analysis of the morphology and spatial distribution of chimneys beneath the Nyegga pockmark field, offshore mid-Norway. Basin Res 22:465–480. https://doi.org/10.1111/j.1365-2117.2010.00486.x

Huuse M, Jackson CAL, Van Rensbergen P, Davies RJ, Flemings PB, Dixon RJ (2010) Subsurface sediment remobilization and fluid flow in sedimentary basins: an overview. Basin Res 22:342–360. https://doi.org/10.1111/j.1365-2117.2010.00488.x

Isiaka AI, Durrheim RJ, Manzi MS, Andreoli MA (2017) 3D seismic analysis of the AK Fault, Orange Basin, South Africa: implications for hydrocarbon leakage and offshore neotectonics. Tectonophysics 721:477–490. https://doi.org/10.1016/j.tecto.2017.10.011

Judd AG, Hovland M (2007) Seabed fluid flow: impact on geology. Cambridge University Press, Cambridge, Biology and the Marine Environment, p 475p

Book   Google Scholar  

Judd AG, Long D, Sankey M (1994) Pockmark formation and activity, UK block 15/25, North Sea. B Geol Soc Denmark 41:34–49

Jungslager EH (1999) Petroleum habitats of the Atlantic margin of South Africa. Geol Soc Lond Sp Publ 153:153–168. https://doi.org/10.1144/GSL.SP.1999.153.01.10

King LH, MacLean B (1970) Pockmarks on the Scotian shelf. Geol Soc Am Bull 81:3141–3148. https://doi.org/10.1130/0016-7606(1970)81[3141:POTSS]2.0.CO;2

Koopmann H, Schreckenberger B, Franke D, Becker K, Schnabel M (2016) The late rifting phase and continental break-up of the southern South Atlantic: the mode and timing of volcanic rifting and formation of earliest oceanic crust. Geol Soc Lond Sp Publ 420:315–340. https://doi.org/10.1144/SP420.2

Koson S, Chenrai P, Choowong M (2014) Seismic attributes and their applications in seismic geomorphology. Bull Earth Sci Thail 6:1–9

Kramer EAJ, Heck L (2014) Orange basin 3D pre-processing and preSDM 2013 shell global solutions international BV. Risjswijk, Netherlands

Krueger A, Gilbert E (2009) Deepwater fold-thrust belts: not all the beasts are equal. AAPG search and discovery article 30085. https://www.searchanddiscovery.com/documents/2009/30085krueger/index.htm . Accessed 9 Nov 2022

Kuhlmann G, Adams S, Campher C, van der Spuy D, di Primio R, Horsfield B (2010) Passive margin evolution and its controls on natural gas leakage in the southern Orange Basin, blocks 3/4, offshore South Africa. Mar Petrol Geol 27:973–992. https://doi.org/10.1016/j.marpetgeo.2010.01.010

L’Arvor E, Maya T, Lattes P (2020) Seismic survey challenges offshore South Africa. In Second EAGE marine acquisition workshop 2020. EAGE Publications BV, pp 1–3

Lebedeva-Ivanova N, Polteau S, Bellwald B, Planke S, Berndt C, Stokke HH (2018) Toward one-meter resolution in 3D seismic. Lead Edge 37:818–828. https://doi.org/10.1190/tle37110818.1

Light MPR, Maslanyj MP, Greenwood RJ, Banks NL (1993) Seismic sequence stratigraphy and tectonics offshore Namibia. Geol Soc Lond Sp Publ 71:163–191. https://doi.org/10.1144/GSL.SP.1993.071.01.08

Loncke L, Mascle J, Parties FS (2004) Mud volcanoes, gas chimneys, pockmarks and mounds in the Nile deep-sea fan (Eastern Mediterranean): geophysical evidences. Mar Petrol Geol 21:669–689. https://doi.org/10.1016/j.marpetgeo.2004.02.004

Løseth H, Gading M, Wensaas L (2009) Hydrocarbon leakage interpreted on seismic data. Mar Petrol Geol 26:304–1319. https://doi.org/10.1016/j.marpetgeo.2008.09.008

Løseth H, Wensaas L, Gading M (2011) Deformation structures in organic-rich shales. AAPG Bull 95:729–747. https://doi.org/10.1306/09271010052

Maduna NG, Manzi MS, Jinnah Z, Bourdeau JE (2022) Strato-structural evolution of the deep-water Orange Basin: constraints from 3D reflection seismic data. Solid Earth 13:1755–1780. https://doi.org/10.5194/se-13-1755-2022

Mahlalela V, Manzi MSD, Jinnah Z, Bourdeau JE, Durrheim RJ (2021) Structural characteristics and 3D seismic detection of gas migration pathways in the deep-water Orange Basin. S Afr Mar Geophys Res 42:8. https://doi.org/10.1007/s11001-021-09428-y

Manzi MSD, Durrheim RJ, Hein KAA, King N (2012) 3D edge detection seismic attributes used to map potential conduits for water and methane in deep gold mines in the Witwatersrand basin, South Africa. Geophysics 77:W133–W147. https://doi.org/10.1190/geo2012-0135.1

Manzi MSD, Hein KAA, King N, Durrheim RJ (2013) Neoarchaean tectonic history of the Witwatersrand Basin and Ventersdorp Supergroup: new constraints from high-resolution 3D seismic reflection data. Tectonophysics 590:94–105. https://doi.org/10.1016/j.tecto.2013.01.014

Maselli V, Kneller B, Taiwo OL, Iacopini D (2019) Sea floor bedforms and their influence on slope accommodation. Mar Pet Geol 102:625–637. https://doi.org/10.1016/j.marpetgeo.2019.01.021

Maslanyj MP, Light MPR, Greenwood RJ, Banks NL (1992) Extension tectonics offshore Namibia and evidence for passive rifting in the South Atlantic. Mar Petrol Geol 9:590–601. https://doi.org/10.1016/0264-8172(92)90032-A

Mazzini A, Etiope G (2017) Mud volcanism: an updated review. Earth Sci Rev 168:81–112

Mazzini A, Nermoen A, Krotkiewski M, Podladchikov Y, Planke S, Svensen H (2009) Strike-slip faulting as a trigger mechanism for overpressure release through piercement structures. Implications for the Lusi mud volcano. Indonesia Mar Pet Geol 26:1751–1765. https://doi.org/10.1016/j.marpetgeo.2009.03.001

Menzies MA, Klemperer SL, Ebinger CJ, Baker J (2002) Characteristics of volcanic rifted margins. In: Menzies MA, Klemperer SL, Ebinger CJ, Baker J (eds) Volcanic rifted margins. Geological Society of America, Boulder, pp 1–14. https://doi.org/10.1130/0-8137-2362-0.1

Mitchum RM, Vail PR, Thompson S (1977) Seismic stratigraphy and global changes of sea level: Part 2. The depositional sequence as a basic unit for stratigraphic analysis: Section 2. Application of seismic reflection configuration to stratigraphic interpretation, AAPG Memoir, Tulsa, pp 53–62

Mohammed M, Paton D, Collier REL, Hodgson N, Negonga M (2017) Interaction of crustal heterogeneity and lithospheric processes in determining passive margin architecture on the southern Namibian margin. Geol Soc Lond Sp Publ 438:177–193. https://doi.org/10.1144/SP438.9

Morley CK, King R, Hillis R, Tingay M, Backe G (2011) Deepwater fold and thrust belt classification, tectonics, structure and hydrocarbon prospectivity: a review. Earth-Sci Rev 104:41–91. https://doi.org/10.1016/j.earscirev.2010.09.010

Naseer MT, Asim S (2017) Detection of cretaceous incised-valley shale for resource play, Miano gas field, SW Pakistan: spectral decomposition using continuous wavelet transform. J Asian Earth Sci 47:358–377. https://doi.org/10.1016/j.jseaes.2017.07.031

Nemčok M, Schamel S, Gayer R (2005) Thrustbelts: Structural architecture, thermal regimes and petroleum systems, 1st edn. Cambridge University Press, Cambridge, pp 1–541

Palan K, Green AN, Andrews B, Sink K, Wiles EA (2020) A morphometric analysis of the fluid flow features of the southern Orange Basin South Africa. Mar Geol. https://doi.org/10.1016/j.margeo.2020.106145

Paton DA, di Primio R, Kuhlmann G, van der Spuy D, Horsfield B (2007) Insights into the petroleum system evolution of the southern Orange Basin, South Africa. South Afr J Geol 110:261–274. https://doi.org/10.2113/gssajg.110.2-3.261

Paton DA, van der Spuy D, di Primio R, Horsfield B (2008) Tectonically induced adjustment of passive-margin accommodation space; influence on the hydrocarbon potential of the Orange Basin, South Africa. AAPG Bull 92:589–609. https://doi.org/10.1306/12280707023

Petroleum Agency South Africa (PASA) (2017) Information and opportunities, Petroleum Agency South Africa, Cape Town, South Africa. https://www.petroleumagencysa.com/images/pdfs/Pet_expl_opp_broch_2017bw1.pdf . Accessed 9 Nov 2022

Planke S, Svensen H, Hovland M, Banks DA, Jamtveit B (2003) Mud and fluid migration in active mud volcanoes in Azerbaijan. Geo-Mar Lett 23:258–268. https://doi.org/10.1007/s00367-003-0152-z

Posamentier HW, Martinsen OJ, Shipp RC (2011) The character and genesis of submarine mass-transport deposits: insights from outcrop and 3D seismic data. Mass-transport deposits in deepwater settings. SEPM Special Publication, Tulsa, pp 7–38

Randen T, Monsen E, Signer C, Abrahamsen A, Hansen JO, Sæter T, Schlaf J (2000) Three‐dimensional texture attributes for seismic data analysis. In: 2000 SEG Annual Meeting, pp 668–671. https://doi.org/10.1190/1.1816155

Roberts HH, Hardage BA, Shedd WW, Hunt J Jr (2006) Seafloor reflectivity—An important seismic property for interpreting fluid/gas expulsion geology and the presence of gas hydrate. Lead Edge 25:620–628. https://doi.org/10.1190/1.2202667

Rogers J, Rau A (2006) Surficial sediments of the wave-dominated Orange River Delta and the adjacent continental margin off south-western Africa. Afr J Mar Sci 28:511–524. https://doi.org/10.2989/18142320609504202

Rommerskirchen F, Condon T, Mollenhauer G, Dupont L, Schefuß E (2011) Miocene to Pliocene development of surface and subsurface temperatures in the Benguela current system. Paleoceanography 26:1–15. https://doi.org/10.1029/2010PA002074

Rowan MG, Peel FJ, Vendeville BC (2004) Gravity-driven fold belts on passive margins. In: Mcclay KR (ed) Thrust tectonics and hydrocarbon systems. AAPG Bull, Tulsa, pp 157–182

Scarselli N, McClay K, Elders C (2016) Seismic geomorphology of cretaceous megaslides offshore Namibia (Orange Basin): Insights into segmentation and degradation of gravity-driven linked systems. Mar Petrol Geol 75:151–180. https://doi.org/10.1016/j.marpetgeo.2016.03.012

Sehoole L, Manzi MS, Zhang SE, Bourdeau JE (2020) An innovative seismic and statistical approach to understand 3D magmatic structures and ore deposits in the western Bushveld Complex. S Afr. Ore Geol Rev 126:103784. https://doi.org/10.1016/j.oregeorev.2020.103784

Séranne M, Anka Z (2005) South Atlantic continental margins of Africa: a comparison of the tectonic vs climate interplay on the evolution of equatorial west Africa and SW Africa margins. J Afr Earth Sci 43:283–300. https://doi.org/10.1016/j.jafrearsci.2005.07.010

Silva CC, Marcolino CS, and Lima FD (2005) Automatic fault extraction using ant tracking algorithm in the Marlim South Field, Campos Basin. Paper presented at the 2005 SEG Annual Meeting, Houston, Texas, 6–11 November 2005, 857–860. https://doi.org/10.1190/1.2148294

van der Spuy D, Sayidini B (2022) Offshore Namibia discovery signals bright future for South Africa oil and gas. AAPG Explorer. https://explorer.aapg.org/story/articleid/62613/offshore-namibia-discovery-signals-bright-future-for-south-africa-oil-and-gas . Accessed 31 Mar 2022

Uenzelmann-Neben G, Tobias W, Gruetzner J, Maik T (2017) Transition from the Cretaceous ocean to Cenozoic circulation in the western South Atlantic— a twofold reconstruction. Tectonophysics 716:22–240. https://doi.org/10.1016/j.tecto.2016.05.036

van der Spuy D (2003) Aptian source rocks in some South African Cretaceous basins. Geol Soc Lond Sp Publ 207:185–202. https://doi.org/10.1144/GSL.SP.2003.207.10

Viola G, Andreoli M, Ben-Avraham Z, Stengel I, Reshef M (2005) Offshore mud volcanoes and onland faulting in southwestern Africa: neotectonic implications and constraints on the regional stress field. Earth Planet Sci Lett 231:147–160. https://doi.org/10.1016/j.epsl.2004.12.001

Viola G, Kounov A, Andreoli MAG, Mattila J (2012) Brittle tectonic evolution along the western margin of South Africa: more than 500 Myr of continued reactivation. Tectonophysics 514:93–114. https://doi.org/10.1016/j.tecto.2011.10.009

Weigelt E, Uenzelmann-Neben G (2004) Sediment deposits in the Cape Basin: indications for shifting ocean currents? AAPG Bull 88:765–780. https://doi.org/10.1306/01260403101

Wildman M, Brown R, Watkins R, Carter A, Gleadow A, Summerfield M (2015) Post break-up tectonic inversion across the southwestern cape of South Africa: New insights from apatite and zircon fission track thermochronometry. Tectonophysics 654:30–55. https://doi.org/10.1016/j.tecto.2015.04.012

Yilmaz Ö (2001) Seismic data analysis: Processing, inversion, and interpretation of seismic data. Society of Exploration Geophysicists, Tulsa

Download references

Acknowledgements

For funding the first author’s doctoral, our sincerest thanks are extended to the National Research Foundation (NRF) of South Africa and the South African Council for Geoscience (CGS), respectively. We would also like to thank Shell and the Petroleum Agency of South Africa (PASA) for providing the 3D reflection seismic data, and Schlumberger for the Petrel software and support. We are furthermore grateful for the scientific discussions and insights provided by our friends and colleagues from the Wits Seismic Research Centre. The authors thank two anonymous reviewers whose comments have greatly improved the manuscript. The authors also thank Dr. Wu-Cheng Chi for editorial handling.

Open access funding provided by University of the Witwatersrand. This study was funded by the National Research Foundation (NRF) of South Africa for the first author’s doctoral under grant UID: 130186, and the South African Council for Geoscience (CGS) for the first author’s master’s degree.

Author information

Authors and affiliations.

School of Geosciences, University of the Witwatersrand, Johannesburg, South Africa

N. G. Maduna, M. S. D. Manzi & Z. Jinnah

Geological Survey of Canada, 601 Booth Street, Ottawa, ON, K1A 0E8, Canada

J. E. Bourdeau

Wits Mining Institute, University of the Witwatersrand, Johannesburg, South Africa

You can also search for this author in PubMed   Google Scholar

Contributions

All authors contributed to the conception and design of the manuscript. NGM conducted the research and wrote the manuscript draft; MSDM and ZJ supervised the main author’s research; MSDM, ZJ, and JEB reviewed and edited the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to N. G. Maduna .

Ethics declarations

Competing interests.

The authors have no competing interests to declare that are relevant to the content of this manuscript.

Ethical approval

Not applicable.

Consent to participate

Consent for publication.

All authors give consent for publication.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 19 KB)

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Maduna, N.G., Manzi, M.S.D., Bourdeau, J.E. et al. 3D reflection seismic imaging of natural gas/fluid escape features in the deep-water Orange Basin of South Africa. Mar Geophys Res 44 , 17 (2023). https://doi.org/10.1007/s11001-023-09523-2

Download citation

Received : 17 December 2022

Accepted : 02 May 2023

Published : 25 May 2023

DOI : https://doi.org/10.1007/s11001-023-09523-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Deep-water Orange Basin
• 3D reflection seismic data
• Gas/fluid escape feature
• PHAA anticline
• Find a journal
• Publish with us
• Track your research