Digital Toolkit to Assist the Interpretation of Traditional Masonry Construction

ABSTRACT

Masonry construction is ubiquitous and is most typically employed for the construction of fundamental structural architectural elements. It is thus extremely important in architectural conservation, and visual survey, recording and documentation often start with such primary elements. Recording has, however, traditionally been a laborious manual process yielding variable and subjective outcomes. This situation is exacerbated by the need to involve experts to interpret the recorded data, which further increases costs. Recent and ongoing advances in digital reality capture and computer vision afford notable advantages in extracting value from point cloud data with benefits to conservative repair, maintenance, and wider architectural intervention. In this paper, we build on our previous research by the authors and present bespoke algorithms brought together as a ‘digital toolkit’ to automatically segment point clouds into individual masonry units and further characterisation of each unit to support architectural interpretation. The use of this digital toolkit is illustrated using a wall section of Linlithgow Palace (Scotland, UK) as a case study, but it must be emphasised that this form of construction typifies innumerable buildings globally. The toolkit has shown to facilitate rapid and systematic identification of change in masonry construction styles, sizes and geometry, and, when used in conjunction with traditional survey evaluation, it meaningfully assists interpretative capabilities. This combined approach is termed ‘digitally assisted analytical recording’ and it offers the promise of yielding more cost-effective, accurate survey for interpretation.

KEYWORDS:

• 3D pointcloud
• digital reality capture
• digital toolkit
• historic digital survey
• masonry interpretation
• shape descriptor
• stone geometry

1. Introduction: the importance of recording for interpretation

It is well understood that documentation and recording for traditional buildings is a primary tenet of conservation practice and is encapsulated in the vast majority of international building conservation frameworks and charters (Bell 1997; BS7913, 2013). Among the many reasons asserted for recording, it principally facilitates interpretation and analysis of structure, fabric, form and function. Recording supports analysis of often complex buildings and architectural assemblages that may have a single constructional period or more frequently have evolved in successive stages that reflect socioeconomic, political, legislative, and technical change. Indeed, without an accurate understanding of these intricate architectural historical overlays, development of sympathetic and targeted conservation treatments cannot be effectively substantiated and, ultimately, implemented (UK ICOMOS, 1990). Architectural recording therefore forms the basis for much of building conservation practice and is, in many cases, the starting point for design and physical intervention. However, recording for recording’s sake is intuitively a missed opportunity and as Dallas (2003, 13) asserts “recording is of limited or reduced value unless analysis leading to better understanding takes place, which in turn should inform decisions regarding repair, alteration and use …. This process of interpretation and analysis should seek to provide a narrative of the building’s history and use”. A clear connection and logical information flow therefore exists between recording, interpretation, attachment of significance and intervention (BS 7913 2013). Indeed, the notion of “analytical recording” (UK ICOMOS, 1990, 43) is set centre stage in the determination of understanding in architectural change and has been defined as “consisting largely in the interpretation and subsequent recording of the significant evidence for a building’s form and subsequent history that is contained within its fabric” — it is fundamentally a “recognition of change (or absence of it)” (UK ICOMOS 1990, 73).

Traditional determination of change and its tell-tale features require a professionally collaborative endeavour relying upon a combination of multiple sources of information (archival, documentary, relative and absolute) and expertise (e.g. historians and an array of built environment professionals). Pragmatic difficulties are however omnipresent with this approach, as each phase of evaluation is associated with greater cost, and therefore attaining better understanding can be prohibitively expensive within the scope of the heritage sector that ostensibly operates within limited budgetary constraints.

In attempts to codify ‘stratigraphic deposition, disturbance and relationships’ (Dallas 2003, 61) via recording, Dallas (2003, 3) indicates that archaeological approaches to interpretation and analysis have used “stratigraphic techniques to demonstrate inter-relationships of phasing in complex buildings”. The formalised, ordered and logical structuring of stratigraphic recorded data is important and as Dallas (2003, 13) indicates, ‘this process assigns a unique context number to every individual feature or building event, such as the main construction phases, the insertion, replacement and enlargement of doorways and window openings’. The principles of archaeological stratigraphy were first developed by Harris (2014) and his concept was considered a fundamentally important advancement in recording as it introduced ‘time’ and the visual representation of ‘stratigraphic sequencing’ to analysis. Such stratigraphy techniques (such as a ’Harris’ matrix) have been classically undertaken manually. However, today, the use of digital reality capture and automatic data processing techniques for data segmentation offer advantages in attaching unique context data labelling. In fact, the first-order ability of attaching accurate unique identification as well as descriptive characteristics to each individual stone is fundamental for a myriad of conservation operations beyond stratigraphic interpretation. The research presented in our paper facilitates the unique labelling of masonry units via automatic segmentation and assisted analytical recording for interpretation operations.

2. Materials and method

2.1. Determining interpretative value from masonry styles and its analysis

Masonry, as a construction material, is seen on a global scale and its ubiquity is reflected in an incalculable number of structures. Historic value and cultural significance encoded in masonry is therefore of critical importance for interpretation and requires great consideration. Indeed, such is its importance that UK ICOMOS (1990, 73) highlights many ’interpretative elements’ to aid those entrusted with the identification of architectural change through the evaluation of masonry substrates. Here, those ‘interpretative elements’ that are directly relevant to masonry construction have been tabulated (see Table 1) to aid clarity.

Table 1. Interpretative elements aiding the identification of change in masonry (UK ICOMOS 1990, 73).

Recognising when materials have been re-used and when possible, identifying their source. Differences in material. Breaks in masonry. Notice should be taken of the precise form of the break (ragged or straight; whether aligned with internal partitions, etc; fabric change or a blocked opening). Differences in structural techniques. There are changes over time in brickwork, construction, etc. Stylistic differences in architecture and decoration. Evidence for the demolition of part of a building. This will often be shown in a masonry building by irregularities at angles or of coursing. In all cases it may be suggested by anomalies in plan, by blocked openings that cannot otherwise be explained, or by the use of a different material in making good the area demolished.

Importantly, the ‘interpretive elements’ give specific guidance relating to assessment applied to change in masonry structure and fabric in attempts to attain architectural understanding for evidencing successive phases of construction. Many of these elements are overtly highlighted in more recent guidance on ‘Metric survey specifications for cultural heritage’ (Bryan et al. 2013). This document reinforces the importance of consistent recording, the required level of detail and completeness of survey in operations. Indeed, of particular relevance is the emphasis upon walling fenestration through to the individual stone masonry units.

Masonry substrates relay primary archaeological and architectural information essential for understanding initial construction and for deciphering change. Within this context, greater interpretative value can be attained if an understanding of traditional stone masonry craft skills and their techniques is adopted. Indeed, recognition of interplay between the geology of the masonry materials and stone reduction processes undertaken by those who built or repaired the structure aid understanding (akin to experimental archaeology analysis processes). A basic recognition of the ability to dress stone from poorly shaped material (i.e. from quarry) to accurately worked dimensional ashlar requires a significant investment in time and skill and is a function of the geological nature of the stone (sedimentary stone vs metamorphic and igneous), its ease of reduction and shaping, as well as the wealth of those commissioning the work. Hence, those materials that tend to ashlar blocks with tight, uniform mortar joints reflect time and effort and are logically associated with greater quality and status of the walling type (Brunskill 1978; Warland 1929). A rudimentary taxonomy of masonry styles (see Figure 1) reflects the major variations that confront the evaluator (Brunskill 1978). Random rubble requires the least dressing of the individual stones, through to highly worked ashlar that is associated with tightly jointed beds and perpends.

Figure 1. Pictorial classification of common masonry styles.

Uniformity of a masonry style can be indicative of a single build phase whilst variation is logically associated with change. Key factors in masonry assessment include the shape of the masonry (e.g. are the masonry units rounded or squared?), the size of the masonry, the bonding and coursing arrangement and the geological composition of the units.

2.2. Towards digitally supported analytical recording

Techniques and equipment to survey, record and ultimately analyse and interpret information have remained relatively unchanged for centuries and ostensibly relied upon visual inspection and detailed hand measured, dimensioned record drawings of elevations, plans and sections (Browning 1996). These act as primary documents and, combined with archival research evidence, they form much of the basis of building analysis. Analysis of masonry substrates has traditionally been relayed in a discursive narrative manner. A written description of the overarching masonry style is relatively easy to establish, but accurate identification of individual masonry units proves more challenging. Therefore, a general sense of the masonry construction is attained, but this fails to achieve the targeted level of reporting required for practical repair intervention. This creates problems in the accuracy and quantification that is required for undertaking conservative fabric repair.

Compounding the problems of describing masonry substrates, undertaking such detailed recording processes is also logically time consuming. This slow process does however facilitate detailed examination and can yield valuable information regarding the history of the building. As UK ICOMOS (1990, 45) indicates ‘close examination of the structure compels one to notice things that before, while the exact measurement of wall alignments, of angles and of wall thickness may raise questions and suggest solutions that one was not previously aware of’. Real-time analysis and interpretation of information during the recording process is of great value allowing the generation and exploration of multiple hypotheses to occur whilst on site (Forster and Douglas 2010). This is argued to be an advantage of ‘slow recording operations’ such as hand measurement and field book recording as the time taken to undertake the operations notionally facilitates interpretation by those undertaking the operation.

In the last few decades, disruptive technologies such as laser scanning and photogrammetry have been revolutionising data collection in terms of accuracy, rapidity, and objectivity. These technologies have however been shown to enable exhaustive, detailed acquisition of information that is inherently safer to attain. However, since the surveyor is no longer meaningfully required to scrutinise the architecture during the digital data acquisition, this may inadvertently have also led to a loss in the formerly highlighted period of overt analysis undertaken during the slower recording process. Opportunities to ensure no loss of interpretative function associated in well-executed ‘slow recording’ can be made via digital tools applied post survey, while exploring the acquired data on a desktop. Indeed, whether through traditional survey or digital capture processes, it is the collation of the features of the substrate that combine to attain robust analytical data into ‘a coherent account of the building history’ (UK ICOMOS 1990, 45).

Recently, Building Information Modelling (BIM) and Digital Twinning (DT) are transforming information management over the entire life cycle of built environment assets, from modern to historic/heritage buildings. Although, as stated by Anton et al (2018), BIM within the context of historic buildings has yet to be fully implemented, significant progress has been made in the field of Heritage BIM (HBIM) and Digital Twinning (HDT) in which an ‘as built’ traditional structure is digitally documented and modelled to support more effective communication, information exchange and decision-making. More specifically, digitisation of masonry walls, formed by thousands of units (i.e., stones), results in more accurate measurements and objective analysis, easing the communication between stakeholders (Logothetis, Delinasiou, and Stylianidis 2015, Anton et al. 2018; Bruno and Roncella 2019; Valero et al., 2018b). These can be used to produce semantically rich building (3D) models (i.e. Asset Information Models (AIM) or Digital Twin (DT)) (Nieto Julián and Moyano Campos 2014; Valero et al. 2018c) that afford further analysis through modelling and simulations, such as Finite Element Analysis (Asteris et al. 2015; D’Altri et al. 2020). Furthermore, the resulting semantically-rich model can be utilised to increase the contextualised retrieval of data. The eased retrieval of all information relative to a given area reduces traditional ‘desktop’ survey and analysis time and, logically, enhances efficiencies (Kumar 2015).

A growing body of research also focuses on the detection of masonry defects in the 2D/3D survey data increasingly using machine learning or deep learning (Idjaton et al. 2022; Kwon and Yu 2019; Musicco et al. 2021; Valero et al. 2019; Wang et al. 2019). The results of defect detection can then be used to enrich the HBIM/HDT to further enhance masonry health monitoring operations.

Whilst modern surveying, HBIM/HDT as well as AI technologies will transform digital documentation and historic/heritage asset management, their potential benefit to interpretation and analysis remains under-represented in academic literature, with most works exploring the use of digital models within what remain essentially manual analysis processes (Martin, Murillo, and Utrero 2019; Nieto-Julián et al. 2022; Pocobelli et al. 2018).

This paper illustrates the advantages of digital documentation and data processing towards the objectification of masonry analysis. Fundamental to this is the concept of ‘digitally assisted analytical recording’ that aims to support interpretation via data processing toolkits for the extraction of ‘interpretation-ready’ information from digital reality capture data.

A digitally assisted analytical recording toolkit has been developed by the authors and its current status and use is presented here. It comprises tools for 1) automated segmentation of individual stones (and mortar) regions in digitised point clouds of masonry (Valero, Bosché, and Forster 2018a; Valero et al. 2020); and 2) the automated characterisation of the geometry of the individual stones (e.g. area, roundness, elongation, angularity). Altogether, the toolkit helps identify common and specific features in masonry materials, establishing relationships or outlier characteristics. The value of the toolkit is illustrated using the case study of a masonry wall from Linlithgow Palace, Scotland, but, as previously mentioned, this form of construction typifies innumerable buildings globally and therefore transcends its Scottish context in application.

3. Theory/calculation

The digitisation of masonry walls and their semantic documentation, i.e. identification of sub-elements and individual stones, underpins masonry recording and analysis. However, the segmentation of digitised masonry into individual stones (and mortar) is a tedious and time-consuming work that can greatly benefit from automation, with algorithms based on image processing techniques.

Different styles of masonry require different approaches for automatically segmenting stone and mortar regions. Thus, a method used for segmenting ashlar units, where orthogonal patterns and regular shapes are present, would look for horizontal and vertical joints on the wall (Acas 2021); whereas the segmentation of random or coursed rubble masonry, with irregular patterns and units (i.e., stones), would entail a more complex approach. Our previous research (Bosche, Forster, and Valero 2022; Valero, Bosché, and Forster 2018a) proposed the use of the 2D Continuous Wavelet Transform (CWT) to detect mortar areas and, as a result, segment individual stones. The developed segmentation tool was made available as a plugin for the open-source freeware CloudCompare (CloudCompare, 2022; Valero and Bosché 2020),¹ commonly used by professionals from different disciplines working with point clouds (i.e. from computer scientists to archaeologists). The main manual input required by the tool is the approximate width of the mortar regions. The accuracy of automatic segmentation tool was evaluated in the previous research (Valero and Bosche, 2020) and a bespoke graphical user interface (GUI) was also presented to enable interaction and manual correction of erroneous segmentation by the automated algorithm. The efficacy of the combined approach was assessed and reported in the previous work (Bosche, Forster, and Valero 2022; Valero et al. 2020).

In this work, our segmentation tool is used for the segmentation of masonry walls. The application of this software to an example section of a rubble masonry wall is shown in Figure 2a, where individual stones are distinctly coloured. These masonry units are subsequently projected onto a plane tangential to its surface, producing a binary orthoimage of the segmentation, as illustrated in Figure 2b.

Figure 2. Automated segmentation of individual stones and mortar regions in an example masonry wall. a) Segmented stones in the 3D point cloud; b) Binary orthoimage obtained after the projection of the segments on the wall plane. Note that, in this work, the metric dimension of a pixel equals 1 cm.

Following the segmentation and projection processes that segment and label all individual stones on a 2-dimensional map (facade orthoimage), various geometric characteristics are calculated automatically that quantitatively characterise the shape of each stone. These include the stone area (A), rectangularity (R), circularity (C) and elongation (E).

Figure 3 illustrates these calculated geometric characteristics and Table 2 details the calculation formulas and explains their value to architectural interpretation.

Figure 3. Illustration of the geometric characteristics calculated for each masonry unit.

Table 2. Stone architectural features measured automatically by the proposed tools and their value to architectural interpretation.

Stone geometric characteristics	Unit	Description/Formula	Interpretation value
Area (A)	m^2	Area of the 2D surface of the masonry unit: $\mathit{A}=n_{\mathit{pixels}}{u}_{pi\mathrm{xel}}^2$ where $n_{\mathit{pixels}}$ is the numbers of pixels that make up the masonry unit in the 2D ortho-image, and $u_{pi\mathrm{xel}}^{\hspace{0.17em}}$ is the metric dimension of each pixel, e.g. 1 cm (in which case $u_{pi\mathrm{xel}}^2$ is 1 cm^2).	Masonry units with relatively larger areas. These may be acting in a structural bonding capacity and/or may also reflect the inclusion of reused masonry components that seem out of place within the masonry substrate style in which they sit (for example sills/lintels etc).
Rectangularity (R)	Unitless [0; 1]	Fraction of the area of the masonry unit to the area of its axis-aligned bounding box in the 2D ortho-image, $\mathit{R}=\frac{{\mathit{n}}_{\mathit{pixels}}}{h_{\mathit{bb}}\times w_{\mathit{bb}}}$ where $n_{\mathit{pixels}}$ is the number of pixels that make up the masonry unit in the 2D ortho-image, and $h_{\mathit{bb}}$ and $w_{\mathit{bb}}$ are the height and width in pixels of the masonry unit’s bounding box. Note: both rectangular and square stones will have rectangularity values of 1.	R tending to 1 is indicative of a stone being ashlar, whether intentionally formerly dressed or associated with naturally bedded stone.
Circularity (C)	Unitless [0; 1]	Ratio of the Area of a masonry unit, A, to the area of a circle with the same perimeter, p, calculated as the number of pixels in the boundary of the surface of the stone.$\mathit{C}=\frac{4\mathit{\pi }\cdot \mathit{A}}{p^2}$	C tending to 1 is indicative of limited or no stone dressing. Circularity is associated with an increase likelihood of construction with field boulders or other rounded masonry materials derived from geological processes.
Elongation (E)	Unitless [0; 1]	This is calculated as the eccentricity of the ellipse that has the same normalised second central moments as the region defined by the masonry unit. The eccentricity of an ellipse is a measure of how circular it is and is calculated as: $\mathit{E}=1-\frac{c}{a}$ where c and a are the lengths of the minor and major axes, respectively, of the ellipse.	E tending to 1 is indicative of elongation. Elongation in masonry units can be a function of dressing but is commonly associated with natural cleavage in the stone bedding. In terms of construction, elongated stone can be associated with well-considered, comprehensive, strong bonding.

4. Results

4.1. Testing the algorithm — case study

Linlithgow Palace is a category ‘A’ (denoting “buildings of special architectural or historical interest which are outstanding examples of a particular period, style or building type”, Historic Environment Scotland 2022) listed structure and is considered one of the most important buildings in the UK. Indeed, it is the ‘earliest surviving Scottish Royal Palace’, built mainly as a ‘pleasure palace’ and dating from 1403. The listing description indicates that the west range probably began by James III in 1488–1513. The range to West built and remodelled of older ranges by Nichol Jackson, 1502–1512 (Historic Environment Scotland 2012). The West elevation is described by McWilliam and Wilson (1978, 296) as having “few architectural pretension” (see Figure 5). The central section is characterised as being composed of three storys with additional vertical levels noted to the end towers at opposing corners. Whilst McWilliam and Wilson (1978, 296) allude to the fenestration being all rectangular and being of various dates, little indication is given for the various insertions and blockings of the walling masonry fabric. Indeed, whilst the fenestration may be of limited aesthetic importance (as opposed to the North elevation with its series of Oriel windows) the Royal Commission on Ancient and Historic Monuments of Scotland (RCAHMS) plans, shown in Figure 4, reveal the underlying importance of the rooms behind. Of considerable note is the configuration of rooms on the first floor, with the ‘King’s Hall, Presence chamber, and the Bed Chamber being most pronounced. Internally, the ground floor is characterised by a series of functional barrel-vaulted masonry structure. This region is the focus of the investigation and application of the digital tool kit for this work (see shaded zone of floor plans in Figure 4).

Figure 4. Historic ground and first floor plans of Linlithgow Palace with ‘Region of Interest’ (ROI, see Figure 6) indicated to left hand side wall highlighted in red. RCAHMS (Canmore archive) (note that Chalmer was old Scots for Chamber).

Figure 5. West Elevation of Linlithgow Palace — point cloud.

The Historic Environment Scotland (2012) ‘Statement of Significance’ highlights that formal gardens were located to the West and the now blocked arch at first floor level corresponds to building access to this from the intersection of the Presence Chamber and the King’s Hall. The infilled area adjoining the Presence Chamber and the King’s Hall is most likely associated with a doorway leading to the former formal gardens (circa 16C). This is reflected in interpretation data published by Historic Environment Scotland (Historic Environment Scotland 2012). The rubble-built relieving arch is clearly visible with a corresponding height relationship with the other fenestration and wall floor heights of the respective rooms. The third floor contains the Queen’s Hall; Presence chamber; and the Queens Bed Chamber. This mirrors the Kings arrangement seen in the 2^nd floor.

4.2. Analysis using the proposed digital toolkit

Figure 5 shows the complete point cloud of the west elevation of Linlithgow Palace, which has a size of 53 m × 19 m. The point cloud reflects the relative plainness and lack of architectural pretensions aforementioned by McWilliam and Wilson (1978). A Terrestrial Laser Scanning device Leica P40 was used for data acquisition, and it was positioned at a four-key locations, delivering a coloured 3D point cloud containing 156 million points, with a spatial resolution of 2 mm, which was subsequently exported with a 5 mm resolution to produce the depth with that resolution (e.g. 5 mm × 5 mm per pixel), which is the resolution we have used in the past to test and validate our segmentation algorithm (other authors like Idjaton et al. (2021) use the same resolution for facade orthophotos for similar reasons). Given the scale of the whole facade, a specific 15 m × 10 m region of interest (ROI) was identified to test and illustrate the use of the digital toolkit. Another reason for restricting the application of the digitally assisted toolkit to the ROI was to facilitate greater depth of analysis when correlating visual survey with the digital shape descriptor tools. Figure 6 illustrates the point cloud after the application of our automatic segmentation algorithm. For this segmentation work, a laptop HP EliteBook (Intel i7—4.7 GHz, 16 GB RAM) was employed. The central white delineated area denotes the ROI that is then shown enlarged in the right-hand side of Figure 6 reflecting the results of the application of our automatic segmentation algorithm.

Figure 6. West Elevation to Linlithgow Palace with applied automatic masonry segmentation. The ROI highlighted in green in shown enlarged.

As illustrated in Figure 7, the ROI has been subdivided into manageable sections or quadrants. The horizontal line section reflects the approximate internal floor level of the first-floor rooms. This most likely remained unaltered but the room connectivity or ‘room flow’ from Presence Chamber to King’s Hall may have been subject to greater change with alteration of internal doorway connecting the rooms. This is reflected in altered fenestration and blocked opening occurring over time.

Figure 7. ROI divided into quadrants for purpose of analysis of aiding interpretation narrative.

The visual survey was undertaken in accordance with the standard survey methodology established in the RICS (Level 3 survey guidance) (RICS 2021). This included a desktop survey requiring analysis of documentation relating the ROI and a visual inspection from ground level evaluating the external surfaces of the building. It must be emphasised that we were not attempting to determine building or materials’ condition and the survey was a targeted assessment of change and assertion of construction form. The method adopted the classical systematic approach of evaluating all elements, and components in a sequential manner. In this case, the ROI was visually evaluated determining and documenting the overarching masonry style, coursing and bonding, masonry unit materials and colour and importantly assessment of fenestration (whether open or now blocked) and straight joints indicative of change. The visual survey attempted to determine features we previously outlined in Table 1.

Table 3 presents the results of an onsite visual survey that relates to the quadrant subdivided ROI of the West Elevation, and Figure 8 to Figure 11 indicate the area, rectangularity, circularity, and elongation of the individual masonry units for this wall section. In the following, we analyse the information provided by Figure 8 to Figure 11 and relate it to the onsite visual survey to assess whether the proposed toolkit would indeed be helpful in assisting more objective and complete surveying.

Figure 8. Area of stones in cm².

Figure 9. Rectangularity of stones.

Figure 10. Circularity of stones.

Figure 11. Elongation of stones.

Table 3. Visual survey description of masonry and architectural arrangement for focussed area of investigation.

ROI	Visual Survey Description
ROI Ground Floor
West range centre — Barrel vault No. 1 (ground floor)	Coursed squared rubble at ground level. Masonry style tending towards random rubble as wall height increases. Uppermost courses returning back to more regularised coursed squared rubble. Lower RHS 4 quoins noted with distinct risbond joint to now blocked opening continuing above for approx. 1.5 m. Lower masonry sections discoloured — most likely salts. Materials: Variable reddish coloured sandstone. Salt discolouration especially pronounced to base.
West range centre — Barrel vault No. 2 (ground floor)	Coursed squared rubble at ground level. Masonry style tending towards random rubble as wall height increases. Upper section becoming increasingly formed of smaller ‘bitty’ stones. Lower masonry sections discoloured — most likely salts. Materials: Variable reddish coloured sandstone. Salt discolouration especially pronounced to base.
ROI First floor
Presence chamber (1^st floor)	Generally, coursed squared rubble. Masonry style tending towards random rubble in selected regions. Large window formed with dressed lintel and profiled and chamfered jambs (RHS). Sill also chamfered. Relieving arch above lintel not extending the full length of lintel. Voussoirs formed in squared rubble masonry — no clear springers. Sizable, blocked opening in masonry characterised by large arch (upper central region). Voussoirs formed in squared rubble masonry — no clear springers. Risbond masonry section running vertically and directly below the arch (as might be expected). Stitching across former jambs and main walling fabric limited — characterised by ‘bitty infill masonry’. Above arch voussoirs (LHS) visible masonry intervention — rebuilt in smaller stones lacking coursing.
Kings hall (First floor)	Sizable blocked opening in masonry characterised by large arch. Voussoirs formed in squared rubble masonry — no clear springers. Risbond masonry section running vertically and directly below the arch (as might be expected). Stitching across former jambs and main walling fabric limited. Large window formed with dressed lintel and profiled and chamfered jambs. Sill also chamfered. Window dressing (jambs and lintel) exhibiting mechanical damage most likely associated with iron work. Relieving arch above lintel voussoirs formed in squared rubble masonry — no clear springers.

Interpretation for ROI relating to stone area (Figure 8):

The ‘area’ shape descriptor directs the evaluator to overt features that are noted in the visual survey across all four quadrants. The darkest red regions are mainly associated with poorly automatically segmented areas. Red to pink areas are clearly indicative of larger stones that are generally more formally coursed — of particular note are those stones that are seen horizontally at the intersection of the ground and first floor (lifts). In addition, larger rybat (jambs) stones are noted in the now blocked former openings in fenestration to the right-hand side (RHS) of the lower section of barrel vault 1. Relatively large stones are also noted in the masonry infill below the arch (centrally and left-hand side (LHS) located). Conversely, the whiter smaller or ‘bitty’ stonework is potentially associated with the selective patch refacing and interface between former openings and infill. In construction terms, these are logically the most difficult regions to tie between infill and jambs of former openings (especially see below arch). Consideration must be given to the practical difficulties of blocking up former openings in buildings and the aim of creating bonding across masonry and obviating risbond joints — these straight vertical joints are problematic in a structural capacity but are also prone to facilitating penetrating dampness along poorly connected sections of masonry. The aim of tying or toothing masonry to effectively stitch across materials is more difficult in random rubble as opposed to more formalised rubble or ashlar materials.

Pronounced white-coloured regions of smaller masonry construction are most prevalent in the left-hand lower section of barrel vault 1 and also pronounced ‘bitty’ stones to the vertical intersections of former jambs to the main arch (LHS) of the Presence Chamber. In addition, the bitty nature of certain parts of the rubble-built elevation would suggest that a harl coat may have once been applied to cover the underlying substrate. This would have hidden and protected the underlying poor masonry quality and also largely reflects architectural finishing practices that pre-dated the ‘Victorian’ (19th century) fashion for ‘scraping’ historic buildings in wider conjectural restoration schemes (Forster, 2010a & b). The ‘area’ shape descriptor function finds commonality with the visual survey and creates greater opportunities for verification of the assertions made (see Table 3).

Interpretation for ROI relating to rectangularity of stones (Figure 9):

The ‘rectangularity’ shape descriptor directs the evaluator to overt features that are noted in the visual survey across all four quadrants. Rectangularity is noted across the ROI and is associated with stone of a wide range of sizes. Strong rectangular stone regions are noted at the interface between ground and first floor approximately at internal vault level at the intersection of barrel vault 1 and the Presence Chamber. The least rectangularity is noted in the regions that are poorly segmented (denoted by large white regions) regions, the infilled arch voussoirs and small infill areas especially the left-hand side of the King’s Hall quadrant. The left-hand window has a higher degree of rectangularity associated with stones adjoining the rybats/window jambs (Lintel, sill and rybats removed from the point cloud) than the right-hand side window which has limited larger dressed stones.

The vertical intersection of the existing masonry and the infill to the arch at the King’s Hall (RHS of infilled arch) is poorly stitched or toothed. This is reflected in the visible scaring and ‘bitty’ infill at risbond intersections. Beyond the poorly connected former jambs, the rectangularity and regularity in coursing increases especially in the centre of the blocking to arch running vertically.

The tool identifies information that was not readily noted in the visual survey insomuch as it highlights the relatively high frequency of rectangular stones across a wide range of stone sizes (large and small units).

Interpretation for ROI relating to circularity of stones (Figure 10):

The ‘circularity’ shape descriptor identifies information that was not readily noted in the visual survey insomuch as it highlights the relatively high frequency of circularity of stones that are smaller in nature. These stones are prominent across the four quadrants but of particular relevance is the use of rounded smaller stone that could be described as pinnings with a greater frequency of occurrence at the intersection of infill and earlier construction phases. Generally, the ‘circularity’ shape descriptor identifies that selected rounder stones that are larger in nature and are distributed across the whole ROI. Larger circular stones are noted in a coursed manner to the upper section of barrel vault 1 and the lower section of the Presence Chamber and form a pronounced coursing effect (see LHS window). A high frequency of circularity is noted in smaller stones. Within the context of random and poorly formed quasi-coursed rubble masonry, these smaller stones play an important role in the stabilisation of larger masonry units that have complex centres of gravity due to their geometric complexity and tendency to ‘rock’ or a propensity to rotate if they become unbound. Smaller circular stones are also noted in areas that are associated with blockings of former openings and ‘bitty’, ill-formed rubble infill. This is logical since the repair of existing masonry facings can be challenging, especially at the risbond interface of two ill-defined and poorly toothed/stitched sections.

Interpretation for ROI relating to elongation of stones (Figure 11):

The ‘elongation’ shape descriptor directs the evaluator to overt features that are noted in the visual survey across all four quadrants. A relatively stronger presence of elongated masonry is noted in horizontal course below the windows interface between barrel vault 1 and Presence Chamber and to the lower central area of the large blocking masonry to the archway. This correlates with the maso’n's understanding of structural performance of the wall at this region that would be subject to the greatest outward thrust from the interval vault (Beckman 1995). This is logical when viewed from a constructional perspective as the need to create ‘true levels’ or ‘lifts’ prior to successive stages of a build are important for stability. Some pronounced elongated masonry is noted to the lower barrel vault 2 section. Smaller elongated stones are noted to the lower region of barrel vault 1 and whilst relatively small they do create a higher degree of coursing and bonding than ‘bitty’ infill masonry materials. Poorly segmented regions are also associated with loss of elongation across the image. Of particular relevance, the larger stonework tends to be more elongated. In general terms, elongated stones confer stability by comprehensively bridging across the perp ends and creating strong bonding patterns.

4.3. Discussion

The functionality of the digital toolkit is shown in the case study to facilitate focus on specific aspects of the masonry giving a general sense of broader constructional trends. For example, the strongest trends relating to stone ‘area’, ’rectangularity’ and ‘elongation’ appeared at the intersection of floor levels and the main infill components of the former central arch. Elsewhere, the intersection of the ground and first floor in line with the internal ‘springing’ for the barrel vaults for the strong rooms are associated with a high propensity to squared uniformly coursed masonry styles that would afford stronger construction through more comprehensive masonry bonding.

As a result, the authors (some of whom are experts in masonry analysis) believe such a tool could certainly help enhance interpretation works. However, while the potential value appears evident, it is recognised that more extended validation is necessary. In addition to this, further developments will be sought, such as improving the automated segmentation functionality, enabling users to combine stone characteristics during analysis, and developing standardised ontologies and linked data models for recording findings with the buildings’ HBIM/HDT models. Indeed, as can be seen in Figure 6, the segmentation tool deployed by the team does not deliver 100% accuracy and errors first need to be manually corrected, which can negatively impact work efficiency. Whilst these issues are recognised, ongoing advances in automated segmentation of masonry data using deep learning (e.g. (Idjaton et al. 2021)) will ultimately reduce the need for such corrections, streamlining the process and enhancing efficiency. Importantly, ontologies and linked data models need to be developed and agreed (ideally at the international level) for recording stone segmentation as well as non-geometric stone information valuable for interpretation in semantically rich HBIM/HDT models. Finally, pragmatic solutions should be further explored on how to integrate those technologies into professional practice, not just to enhance efficiency, accuracy and objectivity, but also to effectively share and archive interpretation analysis results.

5. Conclusions

This research assists users to identify interpretative change in masonry substrates via the development of a digital toolkit that automatically segments dense point clouds of rubble masonry, and computes shape descriptors characterising the geometry of each stone unit. These shape descriptors (area, rectangularity, circularity and elongation) are fundamental to those assisting the evaluation of masonry substrates. The use of such a digital toolkit will become particularly important as onsite evaluation and classical real-time interpretation associated with slower traditional recording methods is more limited in the increasingly rapid digital data acquisition stages of survey operations. Enhanced tools that assist post-digital survey will logically play a greater role in interpretation, creating meaning to the recorded building. It is not envisaged that these technologies replace expert visual evaluation, but they will become a welcome addition for assisting those entrusted with interpretation of change beyond the initial digital recording stage.

The Linlithgow Palace case study adopted for initial testing was shown to be taxing given the complexity and subtlety of much of the rubble masonry change. This reflects the reality of much of traditional masonry construction that confronts the surveyor in reality. However, the selection of shape descriptors provides a stronger, directed view of certain aspects of masonry construction. This is especially pronounced in identifying stronger horizontality of rectangular, larger and elongated coursing, especially at lifts between ground and first floors. Conversely, the identification of smaller, more circular masonry is indicative of rebuilt and repaired sections, and poorly formed and risbond blockings, etc. The ability to select specific shape descriptors within automatically segmented point clouds will undoubtedly yield interpretative potential for digitally enhanced analytical recording and documentation.

While the toolkit and methodology present in this paper are promising, significant further work remains to be conducted to systematically validate the value, possibly adjust the tools (e.g. the range of geometric characterisation shape descriptors), and, very importantly, integrate the tool with the practice of interpretation professionals, including within a digitalised BIM/DT -based process, following methodologies and ideas presented in (Dezen-Kempter et al. 2018, 2020; Jouan and Hallot 2019; Ph et al. 2018), for example.

We recognise that the toolkit needs to be tested beyond the confines of the case study presented. The segmentation and shape descriptor tools should also be tested by those who commonly undertake such analysis processes to enhance the rigours of the digitally assisted toolkit. Finally, areas have been identified where further development and integration with other technologies (e.g. HBIM/HDT) and practice.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the Historic Environment Scotland [HES] as part of the Historic Digital Survey Project (HDS) in collaboration with University of Edinburgh & Heriot Watt University.

Notes

¹ http://github.com/CyberbuildLab/masonry-cc