BioDeepfuse: a hybrid deep learning approach with integrated feature extraction techniques for enhanced non-coding RNA classification

ABSTRACT

The accurate classification of non-coding RNA (ncRNA) sequences is pivotal for advanced non-coding genome annotation and analysis, a fundamental aspect of genomics that facilitates understanding of ncRNA functions and regulatory mechanisms in various biological processes. While traditional machine learning approaches have been employed for distinguishing ncRNA, these often necessitate extensive feature engineering. Recently, deep learning algorithms have provided advancements in ncRNA classification. This study presents BioDeepFuse, a hybrid deep learning framework integrating convolutional neural networks (CNN) or bidirectional long short-term memory (BiLSTM) networks with handcrafted features for enhanced accuracy. This framework employs a combination of k-mer one-hot, k-mer dictionary, and feature extraction techniques for input representation. Extracted features, when embedded into the deep network, enable optimal utilization of spatial and sequential nuances of ncRNA sequences. Using benchmark datasets and real-world RNA samples from bacterial organisms, we evaluated the performance of BioDeepFuse. Results exhibited high accuracy in ncRNA classification, underscoring the robustness of our tool in addressing complex ncRNA sequence data challenges. The effective melding of CNN or BiLSTM with external features heralds promising directions for future research, particularly in refining ncRNA classifiers and deepening insights into ncRNAs in cellular processes and disease manifestations. In addition to its original application in the context of bacterial organisms, the methodologies and techniques integrated into our framework can potentially render BioDeepFuse effective in various and broader domains.

KEYWORDS:

• Non-coding RNA
• deep learning
• neural networks
• RNA identification
• feature extraction
• model performance
• gene regulation
• biological processes

1. Background

In contemporary times, the ubiquity of digital applications has led to the accumulation of a vast volume of biological data, including RNA sequences. RNA is a vital component in the biology of life, requiring sophisticated and efficient approaches for comprehensive analysis and interpretation [1]. These RNA molecules can be broadly classified into two primary groups: protein-coding and non-coding RNAs. Protein-coding RNAs contribute to the translation process, facilitating the synthesis of proteins. In contrast, non-coding RNAs (ncRNAs) predominantly do not participate in protein generation. While some ncRNAs have had their functional roles identified, the biological importance of most is still not fully understood [2]. Recent advances in high-throughput sequencing technologies have led to the discovery of numerous ncRNA molecules, posing new challenges for the identification and classification of ncRNA families [3–6].

ncRNAs play crucial roles in numerous biological processes, and their dysregulation has been linked to the development and progression of various diseases and disorders. Aberrant expression of ncRNAs, such as miRNAs and lncRNAs, is implicated in the initiation, progression, and metastasis of several cancer types, including breast, lung, colorectal, and prostate cancer [7–10]. Dysregulation of ncRNAs has also been associated with neurodegenerative diseases, such as Alzheimer’s disease [11], Parkinson’s disease [12,13], and Huntington’s disease [14], in addition to psychiatric disorders such as schizophrenia [15], bipolar disorder [16], and autism spectrum disorder [17]. Altered expression of ncRNAs is connected to cardiovascular diseases [18], including heart failure [19], myocardial infarction [20], atherosclerosis [21], and hypertension [22]. Moreover, ncRNA dysregulation has been implicated in the development of metabolic disorders, such as diabetes [23], obesity [24], and non-alcoholic fatty liver disease [25]. Additionally, ncRNAs regulate various biological processes in plants, which can indirectly influence their response to climate change [26,27].

The identification of ncRNAs is a challenging task due to various factors, including their diverse structures, functions, and sequences [28]. Additionally, ncRNAs have lower conservation levels than protein-coding genes, making it difficult to detect homologous ncRNA sequences across different species [29]. High-throughput sequencing technologies have facilitated the discovery of novel ncRNAs [30]. However, these technologies can generate a very high amount of data, making it troublesome to accurately distinguish between functional ncRNAs and transcriptional noise [31]. One of the significant issues in identifying ncRNAs is the use of alignment-based tools, which have been primarily designed for protein-coding sequences. These tools may not perform optimally for ncRNAs, as their structures and sequence conservation patterns differ from those of protein-coding genes [32]. Consequently, developing specialized tools tailored for ncRNA identification is necessary to address these challenges. Furthermore, the lack of a clear, universally accepted definition for ncRNAs and the absence of a comprehensive database complicate their identification and classification [33]. Computational tools and algorithms for ncRNA prediction and annotation are being continually developed and refined. However, there is still a need for improvement in sensitivity, specificity, and computational efficiency [34].

Hence, these issues render biological sequence classification a complex task, leading to an increasing demand for innovative techniques and methods that can effectively and efficiently analyse sequences [35]. Machine learning (ML) algorithms have been successfully used for identifying ncRNAs due to their ability to model complex patterns and relationships within large datasets [36]. By leveraging advanced algorithms and statistical techniques, ML can capture the unique characteristics of ncRNAs, such as sequence features and structural motifs, which can be used to distinguish them from protein-coding sequences [37]. Deep learning, a subset of ML, has been recently used with good results in this area. Convolutional neural networks (CNNs) and recurrent neural networks (RNNs), variations of deep learning, enable the automatic extraction of relevant features from raw sequence data, thereby enhancing the predictive accuracy and efficiency of ncRNA identification [38,39].

The classification and analysis of ncRNA sequences have become increasingly important in understanding biological processes and disease mechanisms. Fiannaca, La Rosa, La Paglia, Rizzo, and Urso [40] developed nRC, a non-coding RNA classifier that uses features from ncRNA secondary structure and CNNs trained with deep learning algorithms.

The nRC tool classified with high predictive accuracy data from 13 different ncRNA classes, outperforming existing classification tools. On a related note, Liu et al. [39] constructed a deep learning-based model to effectively distinguish long non-coding RNAs (lncRNAs) from messenger RNAs (mRNAs), which can aid in studying diseases at the molecular level. Their model used k-mer embedding vectors, a bidirectional long short-term memory (BiLSTM) layer, and a CNN layer with three hidden layers to achieve classification performance superior to those obtained by traditional methods, such as PLEK, CNCI, and CPC. Our model’s design, in contrast, supports various forms of sequence encoding, providing versatility and effectiveness in analysing different types of biological sequences. This offers an advantage over models restricted to specific encodings, thus, enhancing our model’s adaptability to varying biological data types and boosting its classification performance.

Chantsalnyam, Lim, Tayara, and Chong [41] introduced ncRDeep, a method for predicting the class of ncRNAs using only RNA sequence information as input and employing a CNN for classification. This approach improved the average accuracy by 8.32% compared to state-of-the-art methods. Furthermore, Chantsalnyam et al. [2] proposed ncRDense, a densely connected CNN architecture that extracts high-level features from RNA sequences and uses additional features, such as secondary structure and nucleotide chemical properties, to improve classification accuracy. This model outperformed existing methods when evaluated on benchmark datasets. Similarly, Liu et al. [39] presented a novel deep learning model that distinguished lncRNAs from mRNAs, achieving high F1-score, accuracy, and AUROC values. This model has the potential to detect and understand diseases associated with lncRNAs, opening up new possibilities for diagnostic and therapeutic approaches.

While deep learning methods have successfully automatically extracted features from data [42], combining external features with deep learning still needs to be adequately explored, as there is significant room for improvement in this area. Notably, Dong, Feng, Zhai, Chang, and Mai [43] achieved an accuracy exceeding 99% in classifying white blood cell images by incorporating handcrafted and deep learning features. Similarly, Chantsalnyam et al. [2] demonstrated a gain of over 1% in accuracy by integrating external features in the classification of ncRNA families. These findings underscore the importance of fusing deep semantic features obtained from deep learning methods with traditional artificial features to enhance the performance of deep learning models. Nevertheless, further research and investigation into combining diverse and additional features in deep learning models are imperative to exploit their potential fully.

To decipher the complex landscape of biological data, we present a hybrid deep learning framework, BioDeepFuse. This framework synergistically integrates either CNN and/or BiLSTM networks with external features to achieve enhanced classification accuracy. Including CNN and BiLSTM networks is a strategic choice based on their proven strengths. CNN recognized for its ability to discern both local and global patterns in data, is well-suited for sequence analysis [44,45]. In contrast, BiLSTM, with its unique ability to capture long-term dependencies in sequence data [46], suits the temporal nature of RNA sequences.

Our approach employs a combination of one-hot encoding, dictionary encoding, and advanced feature extraction techniques to express the input sequences. One-hot encoding captures the compositional information of the sequences by representing nucleotides or amino acids as distinct binary vectors [47,48]. In contrast, dictionary encoding may offer compactness and simplicity by assigning a unique ordinal number to each base or residue, and the encoded sequences can be represented using integers rather than high-dimensional vectors [49,50]. The resultant features from these encoding methods are then incorporated into the deep learning architecture. This enables the model to capitalize patterns within ncRNA sequences. Further, we employ advanced training strategies, such as dropout and batch normalization to enhance the model’s learning capability and generalizability. These strategies mitigate overfitting, one of the primary challenges in training deep learning models.

In this study, we have explored various aspects to substantiate our hypothesis, as outlined below:

• Hypothesis: The integration of advanced deep neural networks, such as CNN and BiLSTM, with feature extraction techniques, can significantly enhance the accuracy of ncRNA classification compared to models induced by traditional machine learning algorithms. This hybrid approach may also be effective in analysing complex and diverse biological sequence data, facilitating an understanding of the biological roles and functions of ncRNA.

Our methodology will enable us to address our Research Question (RQ), and as a result, validate or refute the hypothesis, as delineated below:

• RQ: How effectively can the integration of advanced deep learning techniques, such as CNN and BiLSTM, with feature extraction techniques enhance the accuracy and efficiency of ncRNA classification, thereby aiding in the comprehensive analysis and understanding of diverse biological sequence data?

To evaluate our framework and to support our hypothesis and research question, we utilized both benchmark datasets and real-life laboratory RNA samples extracted using the Infernal tool. The results reflected a remarkable level of accuracy in ncRNA classification, surpassing existing methodologies and solidifying the effectiveness of our hybrid deep learning framework in handling complex ncRNA sequence data. The successful integration of either CNN or BiLSTM with advanced encoding techniques opens promising avenues for future research in ncRNA classification. It enhances our understanding of the biological roles and functions of ncRNA.

2. Materials and methods

In the ever-evolving field of genomics, accurate sequence analysis and characterization are essential for understanding the complexities inherent in biological systems. Our research methodology, illustrated in Figure 1, provides a comprehensive framework for the classification of non-coding RNA (ncRNA). It begins with the ‘Data Collection’, detailing the rigorous processes and strategies we employed to curate a diverse and representative dataset. This is followed by the ‘Handcrafted Features’, which elucidates the domain-specific features we manually curated, harnessing our deep understanding of the dataset. Subsequently, the ‘Feature Encoding’ elaborates on the mechanisms we utilized to transform and represent our data optimally for machine-learning models. Finally, the ‘Proposed Architectures’ section presents the advanced computational models we adopted, combining both convolutional and recurrent neural network structures to achieve precise ncRNA classification. Together, these steps offer a holistic view of our approach.

Figure 1. Process for classifying non-coding RNA (ncRNA). First, we gather a diverse dataset in the “data collection” step. Next, in “handcrafted features”, we pull out specific details from the data. Then, in “feature encoding”, we adjust these details to prepare them for the final step. Lastly, in “classification”, we use modern computer models that combine different techniques to classify RNA.

2.1. Data collection

The methodology employed in training the algorithm involved using an extensive and diversified dataset derived from Bonidia et al. [51]. This dataset consisted of 14 different types of bacteria belonging to seven distinct phyla. This diversity ensured a rich and representative training base, allowing the algorithm to learn and generalize from various bacterial examples. Additionally, the inclusion of various phyla also helped to enhance the robustness of the model, as it exposed the algorithm to different bacterial structures and features, thus contributing to the accuracy and effectiveness of the trained model.

We elected to investigate four well-recognized ncRNA classes: sRNA, tRNA, rRNA, and cis-regulatory element. This decision was driven by their prevalent occurrence within bacterial genomes and their marked significance in analytical scenarios. For instance, tRNA and rRNA transcripts’ presence can cloud the analysis of sRNA samples derived from cytoplasmic total RNA extractions, highlighting the need for precise sRNA prediction [52]. Including the cis-regulatory element not only serves as a counterpoint against other gene classes in Rfam’s hierarchy [40] but also enables baseline benchmarking, validating our model’s efficacy. By embracing this diversity of ncRNA classes, we ensure robustness in our model, catering to both well-known and lesser-known ncRNA types, streamlining classifications in real-world applications, and enabling meaningful comparisons with prior research.

We applied the same genomic pipeline adopted in Bonidia et al. [51] to extract the ncRNA classes using the Infernal tool [53]. We accessed the Rfam Public MySQL Database (version 14.9) to obtain RNA type-associated family lists [54]. Four CM files were generated using the complete Rfam covariance model (CM) and these lists using cmfetch, with one file dedicated to each RNA type. We utilized cmsearch with the Rfam curators’ selected gathering cut-off (GA) value [55] for sequence extraction of RNA types from a given genome. The extraction pipeline is illustrated in Figure 2.

Figure 2. Illustration of the data collection pipeline that uses infernal to extract the desired ncRNA sequences from genomes for training and testing. The ’cmfetch’ function retrieves covariance models from the complete Rfam database, and ’cmsearch’ matches these models against a genome. The output consists of multiple FASTA files, each corresponding to a specific ncRNA class. This method is based on the genomic pipeline in Bonidia et al. [51].

To test our model, we introduced a set of 48 new bacterial genomes that had been freshly sequenced. These genomes come from real-world laboratory RNA and refer to newly extracted RNA sequences from specific experimental conditions in our partner laboratory. These sequences were not part of standard benchmark datasets and had not been classified previously, and their inclusion in our study was to validate the robustness and applicability of our hybrid deep learning framework in real-world scenarios. Using these samples provided a unique challenge and opportunity. Since these sequences were not part of standard benchmark datasets and had not been classified previously, we could evaluate our model’s robustness and generalization capability. The experimental validation of our predictions added an extra layer of confidence in our model’s performance.

This additional data pool was strategically chosen to provide an unbiased and rigorous evaluation of the model’s capabilities. Each genome underwent the identical ncRNA extraction for the sRNA, tRNA, rRNA, and cis-regulatory elements, ensuring consistency in methodology between the training and testing phases. Introducing these new genomes into the evaluation framework facilitated a comprehensive assessment of the model’s aptitude for dealing with diverse and previously unseen biological data. All sequences extracted from these sequenced genomes and the dataset with seven phyla are discriminated in Table 1.

Table 1. Number of sequences used for training and testing.

RNA type	Samples	Training	Testing
sRNA	616	497	119
tRNA	631	581	50
rRNA	567	242	325
cis-regulatory	415	246	169

2.2. Handcrafted features and feature encoding

The principles of handcrafted features and feature encoding are fundamental to the data preparation phase in machine learning applications. The pre-processing stage allows data to be encoded through several methods, including k-mer one-hot, k-mer dictionary, or, in some cases, no encoding is applied, with the emphasis placed solely on feature extraction. Notably, our model is engineered to facilitate the extraction of a broad spectrum of features from the dataset.

Handcrafted features, often referred to as engineered features, are variables deliberately extracted from raw data based on domain-specific knowledge. Their design requires a profound understanding of the dataset and the unique problem at hand [56]. In biological sequence classification, these features tap into the intricate nuances of biological sequences, capturing patterns or characteristics potentially missed by automated models. They emerge from rigorous data analysis and iterative experimentation tailored to enhance the efficacy of a specific ML task. Especially when automated feature learning falls short, handcrafted features become invaluable, allowing domain experts to distil essential information for the task [57,58]

Our work used biological features to make a handcraft feature representation. We used descriptors often used in the literature, among them: Nucleic acid composition (NAC), dinucleotide composition (DNC), trinucleotide composition (TNC), Fickett score, Xmer k-Spaced Ymer composition frequency (kGap), and Open Reading Frames (ORF).

• Nucleic Acid Composition (NAC): This feature descriptor calculates the frequency of each nucleotide (A, C, G, T/U) in a DNA or RNA sequence. It provides a simple yet effective sequence characterization, often serving as a baseline for further analysis [59].

• Dinucleotide Composition (DNC): The DNC measures the frequency of each possible pair of nucleotides (AA, AC, AG, …, TT) in a sequence. The DNC represents RNA sequences based on the frequency of adjacent dinucleotides, essentially pairs of nucleotides. This method retains important information embedded in the RNA sequences or fragments [60].

• Trinucleotide Composition (TNC): Similar to DNC, TNC calculates the frequency of each possible triplet of nucleotides (AAA, AAC, AAG, …, TTT) in a sequence. TNC provides even more detailed information about local sequence features, and it is particularly relevant in protein-coding regions, where each triplet (codon) corresponds to a specific amino acid or a stop signal [61].

• Fickett score: The Fickett score is used to identify coding regions (exons) in a DNA sequence. It is based on the observation that nucleotide usage is typically different between coding and non-coding regions. The score is calculated using position-specific base preferences and the relative frequencies of the four nucleotides (L 62).

• Xmer k-Spaced Ymer Composition Frequency (kGap): This method calculates the frequency of nucleotide patterns with k nucleotides intervening between Xmer and Ymer (fixed-length nucleotide patterns). It provides a way to analyse dependencies between nonadjacent nucleotides, capturing longer-range sequence features [63].

• Open Reading Frames (ORF): ORFs are sequences of DNA or RNA that have the potential to be translated into proteins. ORFs start with a start codon (ATG in DNA or AUG in RNA) and end with one of the three stop codons. Features derived from ORFs, such as their count, length, and position, can provide important clues about the protein-coding potential of a sequence [64].

In contrast, feature encoding is a process that converts raw data into a format more conducive to processing by ML algorithms. This could encompass relatively simple transformations, such as transforming categorical variables into numerical values through one-hot encoding. Alternatively, more complex techniques, such as k-mer embedding, can map subsequences of length ’k’ into a continuous vector space [65]. The central aim of feature encoding is to represent the data in a way that streamlines pattern recognition and prediction generation for machine learning models [66].

Within the realm of genomic data, one-hot encoding is a widely used approach for representing nucleotide sequences in ML models [47,48]. This encoding method establishes a direct correspondence between nucleotides and vectors. Each nucleotide is represented by a four-dimensional vector, where only one element is nonzero, indicating the specific nucleotide at a given position. In contrast, all other elements are set to zero. For example, Adenine (A), Cytosine (C), Guanine (G), and Thymine/Uracil (T/U) are encoded as [1, 0, 0, 0], [0, 1, 0, 0], [0, 0, 1, 0], and [0, 0, 0, 1], respectively. On the other hand, dictionary encoding assigns ordinal numbers (ranging from 0 to 3 for bases) to represent each base, providing an alternative mapping approach [49,50]. Additionally, the k-mer representation can be combined with these encoding methods. The representation per se involves breaking down sequences into subsequences of length k. For example, the sequence ’ATGC’ can be segmented into 2-mers, resulting in ’AT’, ’TG’, and ’GC’. Each unique k-mer is then treated as a feature. In the case of one-hot encoding with k-mer (k-mer one-hot), binary vectors are created for each k-mer. In contrast, for dictionary encoding with k-mer (k-mer dictionary), each k-mer is mapped to a numerical value. Combining these representations with both encoding methods has been shown to be effective in deep learning models, as demonstrated in studies by Deng et al. [65]; Jing et al. [49]. Figure 3 provides an illustration of the encoding process for a sequence.

Figure 3. Illustrative depiction of feature encoding techniques for genomic data, focusing on the integration of one-hot encoding, dictionary encoding, and k-mer representation. A hypothetical nucleotide sequence is employed to demonstrate these methods. For one-hot encoding, each nucleotide is mapped to a unique four-dimensional binary vector, while ordinal numbers are used to represent each nucleotide in dictionary encoding. The figure also presents examples of k-mer one-hot and k-mer dictionary encoding, where sequences are divided into subsequences of length ’k’ and encoded using the respective methods.

2.3. Proposed architectures

Our approach to classifying biological sequences is based on the innovative combination of CNNs and LSTM neural networks, leveraging their respective abilities to identify spatial and temporal patterns in complex data. This ML pipeline is designed to work with various forms of sequence encoding, such as k-mer one-hot and k-mer dictionary.

In Figure 4, we can see the developed pipeline. This begins with the data loading phase, where the training and test sets are prepared. At point A of Figure 4, the sequences are appropriately encoded and then undergo the feature extraction process. The selection of encoding methods is flexible, allowing researchers to choose the most suitable approach for the specific type of data they are analysing. After this encoding step, features that are most relevant to the learning task are extracted.

Figure 4. Schematic representation of the proposed ML pipeline for biological sequences classification, integrating CNN and BiLSTM networks. The pipeline initiates with the data loading phase and proceeds with sequence encoding using methods such as k-mer one-hot and k-mer dictionary. Following resource extraction, data is fed into a hybrid deep-learning model comprised of convolutional and LSTM blocks. To accommodate k-mer dictionary encoding, the data is passed through an embedding layer, while convolutional layers extract spatial patterns and LSTMs capture temporal dependencies. Dropout layers are optionally included to prevent overfitting. After processing through LSTMs, data is flattened and directed to fully connected layers for final output generation. The pipeline can integrate multiple encodings and concatenating handcrafted features, allowing for efficient classification of biological sequences. The model parameters and structure were set based on prior empirical work and related literature.

Once the data is preprocessed, it is inputted into a deep learning model comprising a series of convolutional and LSTM blocks. Notably, to facilitate k-mer dictionary encoding, the data undergoes an initial step of being fed into an embedding layer with the dimension of the dense embedding set to 128. The convolutional layers, which may vary in number based on the input parameters, are specifically designed to capture spatial patterns within the sequence. Each convolutional block consists of a 1D convolutional layer equipped with 128 filters and a kernel size of 3. Additionally, optional batch normalization, an activation function (such as ReLU or Leaky ReLU), and a max pooling layer are included in each block. Subsequently, the extracted information is forwarded to the LSTM layers, composed of 128 nodes, and tasked with identifying and learning long-term dependencies within the sequence. Finally, dropout layers can be applied at the end of each block to mitigate overfitting.

After the LSTM stage, the data can either be routed to point C of Figure 4 if utilizing only the CNN architecture or to point B of the same figure if applying the CNN-BiLSTM approach. To enhance both the flexibility and efficacy of the pipeline, each LSTM block can incorporate bidirectional LSTM layers and additional dropout blocks as required. The data are then flattened into a one-dimensional representation, which is fed into fully connected layers to produce the final output. This final step allows the model to generate a compact and insightful representation of the input data, simplifying the subsequent classification task. With such a robust and adaptable pipeline, the efficient classification of biological sequences becomes a practical and potent endeavour.

The model provides a valuable capability for concatenating (Figure 1 at point D) handcrafted features and integrating multiple encodings to infer effectively from diverse representations of biological sequences. This approach leads to the formation of multiple branches, each exploring a unique combination of inputs. The concatenation process can be executed immediately after the data is flattened (Concat I) or after incorporating a dense layer at the end of each branch (Concat II). The selection between direct concatenation and concatenation after a dense layer bears considerable consequences in terms of the model’s performance and interpretability. In practice, in this order, we have at least two dense layers with 128 and 64 nodes before the dense layer with a softmax activation function.

We apply the softmax activation function in the final layer for prediction purposes. The Adam optimizer is used with a learning rate of 1 × 10⁻⁴, while the loss function employed was categorical cross-entropy. We chose a batch size of 32 and trained the model of 100 epochs. To address overfitting, we employed early stopping during training. This technique involved monitoring the test loss over the previous 20 epochs and halting the training process if no reduction in the test loss was observed. As a result, the model’s weights were restored to the best-performing configuration in such cases. The training samples were partitioned into training and validation sets using a ratio of 9:1, preserving the percentage of samples for each class. We used the Keras library to implement the model, and we set the layers and parameters empirically and based on works such as Chantsalnyam et al. [2]; Jing et al. [49]; Noviello, Ceccarelli, Ceccarelli, and Cerulo [67].

3. Results and discussion

To ensure a comprehensive evaluation and meaningful comparison of our proposed model, we conducted experiments using classification models induced by conventional machine learning algorithms: Support Vector Machines (SVMs) and Extreme Gradient Boosting (XGBoost) [68] using only the handcrafted features. We used only handcrafted features for these algorithms, chosen for their robustness when dealing with biological data, which is known to be high-dimensional [69,70], and XGBoost being known to outperform deep models on tabular data [71].

We maintained a consistent training and validation ratio with the deep learning models by employing a 10-fold cross-validation approach. For a fair comparison, we fine-tuned the hyperparameters of these classifiers using the Optuna library [72]. Our optimization targeted the best-weighted precision average across the folds. We performed 100 trials with Optuna, matching the number of epochs used for the deep-learning models.

For SVM, we utilized the radial basis function (RBF) kernel while varying the regularization parameter (C) and the kernel coefficient (γ). As for XGBoost, we explored various hyperparameters, including the maximum depth of a tree, learning rate, number of gradient-boosted trees, minimum sum of instance weight needed in a child, minimum loss reduction required to make a further partition on a leaf node of the tree, subsample ratio of training instances, subsample ratio of columns when constructing each tree, L1 regularization, and L2 regularization. Finally, we evaluated these models using the separate test set and obtained weighted precision scores of 0.8953 and 0.9189 for SVM and XGBoost, respectively.

We thoroughly evaluated our proposed model through a series of experiments. We began by categorizing our experiments into two neural architectures: CNN (without the LSTM block) and CNN-BiLSTM (with a Bidirectional LSTM block). Then, to assess the impact of convolutional layers on classification performance, we varied the number of convolutional blocks used. Regarding the concatenation process, we explored two approaches. The first approach involved direct concatenation (Concat I), while the second employed a dense layer for each branch before concatenation (Concat II). We carefully considered these concatenation methods with handcrafted features during the experiments.

Additionally, we investigated the influence of the encoding method on performance. Specifically, we compared k-mer one-hot (Enc I) and k-mer dictionary (Enc II). Finally, we combined these encoding methods through concatenation to explore potential synergistic effects and examined whether significant improvements could be achieved. Table 2 presents the outcomes of utilizing the CNN architecture.

Table 2. Weighted precision results obtained with different variants of the CNN architecture. The models are categorized based on the encoding method (k-mer one-hot (enc I), k-mer dictionary (enc II), and their combination (enc I + enc II)), the number of convolution blocks, and the concatenation approach (direct concatenation (concat I) and concatenation with a dense layer for each branch (concat II)). Results are compared for 1-mer, 2-mer, and 3-mer configurations. Results surpassing those of traditional classifiers are highlighted in bold.

		Enc I			Enc II			Enc I + Enc II
	Conv Blocks	CNN	Concat I	Concat II	CNN	Concat I	Concat II	CNN	Concat I	Concat II
1-mer	1	0.8602	0.8770	0.8584	0.8746	0.8826	0.8647	0.8682	0.8708	0.8938
	2	0.8738	0.8899	0.8909	0.8775	0.8615	0.8938	0.5009	0.8137	0.9051
	3	0.8822	0.9031	0.8955	0.8850	0.8684	0.9176	0.8872	0.8514	0.9052
	4	0.9077	0.8736	0.9179	0.9261	0.9326	0.8967	0.8822	0.8814	0.9051
2-mer	1	0.8505	0.8285	0.8968	0.8672	0.8079	0.8159	0.8186	0.8471	0.8390
	2	0.8304	0.8382	0.8510	0.8093	0.8390	0.8773	0.8558	0.6307	0.8523
	3	0.8729	0.8331	0.8966	0.8884	0.8801	0.9184	0.8206	0.8471	0.9123
	4	0.8504	0.8716	0.8473	0.8921	0.8736	0.8841	0.8778	0.9027	0.8824
3-mer	1	0.8585	0.8542	0.8743	0.8806	0.8766	0.9032	0.8921	0.8389	0.8894
	2	0.8835	0.8747	0.8934	0.6398	0.8317	0.8658	0.6115	0.6343	0.8897
	3	0.8902	0.8382	0.8817	0.8715	0.8462	0.8592	0.8604	0.8937	0.8912
	4	0.8837	0.8528	0.8970	0.9224	0.8873	0.8676	0.8550	0.8846	0.8776

Table 2 demonstrates a crucial observation: incrementing the number of convolutional blocks enhances the model’s performance. This improvement is evident in both CNN models, with and without concatenation, using handcrafted features. However, fixed convolutional block numbers across varying k-mer lengths do not necessarily yield improvements.

Furthermore, we observe the effectiveness of adopting k-mer dictionary, which consistently produces superior results across multiple experiments. When a CNN model is used without concatenation, k-mer dictionary emerges as the optimal encoding method. On the other hand, concatenation with different encoding methods did not significantly improve the performance and even worsened the performance by a considerable amount in some cases.

Figure 5(a) demonstrates the efficacy of concatenation with handcrafted features across various scenarios, irrespective of the specific concatenation approach employed. Notably, using a k-mer dictionary with direct concatenation yielded the highest weighted precision of 0.9326, outperforming the optimized XGBoost algorithm by 1.37%. Even though certain classes, such as rRNA, exhibit a reduced level of complexity in terms of classification, as depicted in Figure 5(b), the illustration reveals a substantial improvement in the prediction of sRNA and cis-regulatory element sequences when employing deep-learning techniques in conjunction with handcrafted features.

Figure 5. Comparative performance analysis of our proposed architectures against traditional machine learning classification models (SVM and XGBoost), considering the best configurations obtained. (a) Performance comparison of the algorithms using the weighted average of metrics such as precision, recall, and F1-score. The results show consistency throughout the metrics and better performance for CNN with external features. (b) Performance comparison by class using the precision obtained with the algorithms. Even though classes such as rRNA seem to be classified effortlessly, there are clear improvements in precision in classes such as sRNA and cis-regulatory elements.

Although direct concatenation yielded the highest weighted precision, several outcomes indicate the potential for more consistent results by incorporating a dense layer before concatenation. This shows how this approach may enable the model to actively learn complex relationships among the features before merging them, thus enhancing the overall representation power. This approach was adopted by works such as Dong et al. [43], while others such as Chantsalnyam et al. [2] used direct concatenation. Considering how these approaches may work, researchers should carefully consider the trade-offs and select the most appropriate concatenation method based on the task and dataset.

Afterwards, applying the same experiments for the CNN-BiLSTM architecture, we obtained the results shown in Table 3.

Table 3. Weighted precision results obtained with different variants of the CNN-BiLSTM architecture. The models are categorized based on the encoding method (k-mer one-hot (enc I), k-mer dictionary (enc II), and their combination (enc I + enc II)), the number of convolution blocks, and the concatenation approach (direct concatenation (concat I) and concatenation with a dense layer for each branch (concat II)). Results are compared for 1-mer, 2-mer, and 3-mer configurations. Results surpassing those of traditional classifiers are highlighted in bold.

		Enc I			Enc II			Enc I + Enc II
	Conv Blocks	CNN-BiLSTM	Concat I	Concat II	CNN-BiLSTM	Concat I	Concat II	CNN-BiLSTM	Concat I	Concat II
1-mer	1	0.8797	0.8700	0.8864	0.8759	0.8700	0.8700	0.8407	0.8986	0.8847
	2	0.9020	0.8732	0.8639	0.8791	0.8929	0.8726	0.8833	0.8733	0.9062
	3	0.8375	0.9025	0.8875	0.8661	0.9157	0.9247	0.8430	0.9037	0.8924
	4	0.8856	0.9071	0.9109	0.8905	0.9093	0.9110	0.9061	0.9079	0.9006
2-mer	1	0.8839	0.8764	0.8896	0.8916	0.8880	0.8560	0.8906	0.8790	0.8935
	2	0.8791	0.8843	0.8179	0.8638	0.8732	0.8722	0.8669	0.8948	0.9047
	3	0.9018	0.8649	0.8912	0.8670	0.8828	0.9042	0.8966	0.8852	0.8440
	4	0.8528	0.8924	0.9046	0.8591	0.8733	0.8982	0.8210	0.8999	0.9010
3-mer	1	0.8799	0.8544	0.8695	0.8596	0.8713	0.8403	0.8993	0.8583	0.8640
	2	0.8849	0.8810	0.8587	0.8849	0.8571	0.8730	0.8904	0.8568	0.9097
	3	0.8781	0.8762	0.8643	0.8469	0.8756	0.8860	0.8359	0.8868	0.8712
	4	0.7749	0.8559	0.8624	0.8315	0.8906	0.8804	0.7811	0.8901	0.8865

Several notable similarities emerge when comparing CNN with CNN-BiLSTM. First, it becomes apparent that increasing the k-mer size does not necessarily result in improved performance, and incorporating external handcrafted features demonstrates significant gains in multiple scenarios. Specifically, the highest weighted precision achieved was 0.9247, using a k-mer dictionary with concatenation by applying dense layers at the end of each branch. While this value falls below the precision achieved with the CNN architecture, it underscores the crucial role of integrating external features to attain optimal performance.

In contrast to CNN, the CNN-BiLSTM architecture has fewer convolutional blocks and was shown to improve predictive performance in several scenarios. CNNs excel at capturing local patterns and hierarchical representations through filter convolutions applied to the input. Nevertheless, the network’s complexity escalates as the CNN’s depth increases, rendering it susceptible to overfitting. When combined with the BiLSTM block, this increased complexity may explain the observed decline in performance.

Furthermore, it is important to note that the CNN-BiLSTM architecture introduces an additional computational cost due to including the BiLSTM layer. The bidirectional nature necessitates processing the input sequence in both forward and backward directions, resulting in a higher number of computations than the unidirectional LSTM or the CNN alone. This increased computational complexity can pose challenges regarding training time and resource requirements.

Considering the results obtained in our study, where the highest weighted precision was achieved using the CNN architecture with external handcrafted features, it becomes evident that the simpler CNN model may be a more practical and viable choice. Moreover, the computational efficiency of the CNN allows for faster training and inference times, making it a more efficient option for similar applications.

Regarding computational efficiency, the different space complexity for working with the encoding methods proposed are notable. k-mer one-hot needs 4^k × L numbers to properly represent a biological sequence, while k-mer dictionary needs L − k + 1, where L is the sequence length and k is the k-mer size. Despite its more concise representation, the k-mer dictionary encoding consistently yielded the highest weighted precision for both architectures. This shows how researchers can accurately represent biological sequences by employing the k-mer dictionary approach while minimizing memory requirements.

Our research employed Infernal as a fundamental tool for collecting ncRNA sequences derived from bacterial genomes. Infernal is known for its reliability and capacity to discern RNA sequences based on secondary structure features [53]. However, we chose to focus exclusively on primary structure features in our classification approach. The results of our approach demonstrated a consistently robust performance in classifying the obtained sequences, highlighting the effectiveness of our primary structure-based classification methodology.

Although BioDeepFuse was initially conceived with a primary focus on bacterial data, it is essential to recognize that our framework’s foundational methodologies and techniques can extend to other domains, contingent upon the availability of pertinent training data. One salient limitation deserving explicit acknowledgement pertains to BioDeepFuse’s applicability to eukaryotic organisms. This limitation arises from the distinctive complexities inherent in eukaryotic genome annotation [73].

4. Conclusion

In this study, we developed into the exploration of two architectures: the simpler CNN (without LSTM block) and the more intricate CNN-BiLSTM (with a Bidirectional LSTM block). We revealed how variations in the number of convolutional blocks and the incorporation of handcrafted features substantially impact model performance. This insight uncovers valuable directions for future optimizations and research.

Furthermore, we investigated two encoding methods, k-mer one-hot and k-mer dictionary, and assessed their combination using various concatenation strategies. The k-mer dictionary encoding method consistently outshone its counterpart, establishing itself as an efficient tool for representing biological sequences with fewer memory demands. This critical finding emphasizes the substantial role of encoding methods in model design and performance, indicating an area ripe for further exploration.

Our exploration extended to integrating a dense layer before concatenation, aimed at enhancing the model’s capability to discern intricate relationships among features. Although the impact on performance wasn’t striking in this study, this approach still showed potential, underscoring that minute adjustments in model design can yield significant learning benefits.

Moreover, our results indicated that the simpler CNN model surpassed the more complex CNN-BiLSTM regarding weighted precision. This outcome underlines the importance of balancing model complexity and computational efficiency, and it points to the need for careful selection and fine-tuning of model architectures in accordance with task-specific requirements and resource constraints.

Moving forward, our next steps involve automating the presented methodologies, aiming to establish an end-to-end pipeline by leveraging AutoML. This transition would further increase efficiency and ensure the entire process can be executed with minimal manual intervention. Moreover, we plan to extend our research scope into multi-omics approaches, exploring metagenomic and transcriptomic data to better comprehend the myriad intricacies of biological information.

This paper offers insights that can significantly contribute to the handling of biological data by ML algorithms. The results obtained by rigorous experimentation underscore the importance of each component within a predictive model and lay a strong foundation for future advancements in this area.

Availability of data and materials

The documentation, pipeline, images, and results from the models are available in the GitHub repository: https://github.com/brenoslivio/BioDeepFuse. The complete set of 48 new bacterial genomes (and their sequences in FASTA format) are available on the long-term data archive at the Helmholtz Center for Environmental Research – UFZ data centre using the link (https://www.ufz.de/record/dmp/archive/14024).

Acknowledgments

The authors would like to thank USP, CAPES, CNPq, FAPESP, AI4PEP, IDRC, FEMS, ARIS, and HIDA for the financial support for this research. We also thank Denny Popp for his assistance in acquiring the data used in the study.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This project was supported by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior [CAPES] - Universidade de São Paulo [USP]; grant [#2023/00264-0], São Paulo Research Foundation [FAPESP]; Canada’s International Development Research Centre [IDRC] - Grant No.109981; and HIDA – Helmholtz Information and Data Science Academy. The work performed by AVS was supported by a FEMS (Federation of European Microbiological Societies) research and training grant and the Helmholtz Information & Data Science Academy visiting research grant. The work performed by PS and IMM was supported by ARIS project J1-4411.