Exploring the health care system’s representation in the media through hierarchical topic modeling

Abstract

As a large social structure, the health care system is often reflected in media publications. This creates a significant impact on society’s attitude towards the system and the state in general. In order to predict and correct state policies, media actions, and identify media shortcomings, it is necessary to analyze the image portrayed by the media and the public’s attitude towards it. In this article, we present the results of a multidirectional analysis of a corpus of media publications related to health care. We propose a method for analyzing the information image of health care formed by the mass media based on a topic model of a text corpus. The method evaluates reader interest in various healthcare topics, the dynamics of changes in publication sentiment, and the main information trends. The article presents the results of analyzing a corpus of mass media publications in Kazakhstan from January 2020 to January 2023.

Keywords:

• Health care
• mass media
• natural language processing
• topic modeling

Reviewing Editor:

• Chen Jenhui, Full Professor, Computer Science and Information Engineering, Chang Gung University, No. 259, Wen-Hwa 1st Road, Kweishan, Taoyuan, Taiwan

Subjects:

• Technology
• computer science, artificial intelligence, computer science (general), computing & IT security

1. Introduction

Relatively low efficiency and productivity within the health care sector provoke an increase in fiscal restrictions, which in turn contribute to an increase in social tension and a decline in economic growth, especially during the periods of pandemics (Baldwin & Weder di Mauro, 2020). At the same time, the health care system is not only the medical system; it is the part of the social and economic structures of the society. The prevalence of economic restrictions leads to an increase in problems caused by high societal expectations and increased demand for health care services (Atun, 2015). Moreover, insufficient or ineffective representation of health care authorities in the information space contributes to the spread of rumors and misinformation (Tasnim et al., 2020), affects the mental health of the population (Gao et al., 2020; Giri & Maurya, 2021; Aslam et al., 2020; Hamidein et al., 2020). In cases when readers do not have knowledge and experience regarding what is happening, they are especially dependent on the information provided by the mass media (Miller, 1998). Ultimately, this situation leads to such a phenomenon as infodemic, when rumors and false information spread faster than viruses (Pichai, 2023).

Obviously, it is necessary to assess and monitor the extent to which information topics related to health care are interesting for the society, which of these topics demonstrate rapid growth, how the state policy in the field of health care is reflected in mass media publications, etc. In other words, it is important to carry out a systematic multidirectional analysis of the information image of the health care system formed by the media. There are quite a few natural language processing tools for solving this task (Barakhnin et al., 2018; Sadovskaya et al., 2021).

One of such tools is topic modeling. Topic modeling is a method based on the statistical characteristics of the collections of documents, which is used in tasks of automatic abstracting, information extraction, information retrieval and classification (Mashechkin et al., 2013). The method is based on a clearly expressed pattern, reflecting the fact that the corpus of documents forms groups in which the frequency of occurrence of words or combinations of words is different. In this research topic modeling is used to identify the texts related to health care, the topics, which are mostly interesting for readers, information trends that describe health care, the sentiment of publications, and the coverage of public health policy. The directions of analysis of the information image of health care in the study can be illustrated by the Figure (see Figure 1).

Figure 1. Multi-vector analysis of the information image of health care.

Reader interest reflects the preferences of readers for specific topics or themes over a certain period of time. Information trends reflect changes in the media’s coverage of the most important news over time. Illumination of public policy shows how fully the media take into account the government documents aimed at regulating the health care system. Sentiment illustrates media attitude towards health care related topics. Taken together, the above vectors provide several complementary perspectives on the work of the mass media describing the health care system.

The main contribution of the study is as follows.

First, we have proposed a method that allows us performing a multidirectional analysis of the information image of health care formed by the mass media using a topic model for the corpus of texts.

Second, the reader interest, the dynamics of changes in the sentiment of publications and the main information trends are evaluated on the basis of the proposed method.

Third, the method has been tested and the results of its full-scale application on the basis of the corpus of the mass media publications in Kazakhstan are presented.

The study consists of the following sections:

In the first section the authors consider the literature sources that address the relevant topic of the study.

The second section describes the data collection system, the corpus of texts and the hierarchical topic model, with the help of which the media analysis was performed.

The third section describes and discusses the obtained results.

The conclusion summarizes the results, analyzes the limitations of the method and possible future research.

2. Related works

The influence of the media plays an important role in shaping health policy (Baba et al., 2017). For example, an integrated systematic review (Shen et al., 2021) provides an assessment of the impact of planned media interventions, including social networks, on the health policy-making process. The media have a positive impact when their tools are used to improve the accountability of government, to identify priorities, to initiate discussions and to raise awareness (Bou-Karroum et al., 2017). In times of social upheaval, the role of traditional media increases and those people who are usually indifferent to information start using again the traditional sources of news (Casero-Ripolles, 2020). At the same time, the media use special techniques and manipulation mechanisms to form a certain opinion and attitude (Bushman & Whitaker, 2012; Stacks et al., 2015). An infodemic occurred during a global health crisis, leading to the spread of false information and recommendations that were exchanged by media outlets without proper verification and review by specialists (Mheidly & Fares, 2020). News headlines became mostly negative (51% of negative and only 30% of positive ones) (Aslam et al., 2020). The number of publications with a negative sentiment regarding the health care sys-tem has sharply increased (Yakunin et al., 2021). As a result, such information background led to the growth of negative emotions (Hamidein et al., 2020). Due to the great influence of the media, it is necessary to assess the negative impact of media systems and to encourage its positive impact (Bushman & Whitaker, 2012). Properly adjusted public health and media policies can help to increase the public confidence in both traditional media (Tandoc, 2018) and the health care system as a whole. Media monitoring tools, which are part of reputation management (Bauer & Suerdem, 2016; Barysevich, 2019) are used to evaluate the media. However, these systems are inherently limited in evaluation criteria (usually sentiment assessment) and methods (usually keyword search) (Mukhamediev et al., 2020).

From the point of view of monitoring the publications devoted to health care, the literature provides examples of solving the individual problems, such as the sentiment of publications (Aslam et al., 2020), detection of misinformation and rumors (Song et al., 2021), attitudes towards the government social policy in the field of public health (Jo & Chang, 2020; Mohamed Ridhwan & Hargreaves, 2021), and analysis of information trends (Yakunin et al., 2022).

Many works have used sentiment analysis as a tool to study the reactions of people during a pandemic through their social media posts (Dubey, 2020), to identify differences between people’s reactions in different countries (Li et al., 2020), and to assess the dynamics of people’s sentiment (Zhou et al., 2021; Yin et al., 2020). Various techniques are used to extract relevant posts (tweets), including those based on machine learning. For example, recurrent neural network was used in (Nemes & Kiss, 2020) to extract posts related to the pandemic. It seems useful to compile the disparate solutions in the form of a system that allows assessing the publication activity from several perspectives.

However, a sufficiently accurate assessment of sentiment or policy requires a substantial corpus of texts marked up by experts. In the absence of a marked corpus of texts, it is productive to use cluster methods, a variation of which is topic modeling. Topic modeling is widely used in the tasks of automatic referencing, information extraction, information retrieval and classification (Vayansky & Kumar, 2020; Yakunin et al., 2021) and identification (Kirill et al., 2020). The basis of modern topic models is the statistical model of natural language. Probabilistic topic models describe documents by a discrete distribution on a set of topics, and topics by a discrete distribution on a set of terms (Vorontsov & Potapenko, 2012). As a result, the topic model determines which topics each document belongs to and which words form each topic. Groups of terms and phrases formed in the process of topic modeling, in particular, allow solving the problems of synonymy and polysemy of terms (Parhomenko et al., 2017). Topic modeling using various algorithms is widely used to analyze health-related topics (Erzurumlu & Pachamanova, 2020; Rajendra Prasad et al., 2019) and especially COVID-19 (Yin et al., 2022).

In the present study the topic modeling methods are used to solve the problem of analyzing the social impact of the health care system. Specifically, the following tasks were set and solved:

• Ranking of topic groups of media texts (topics) devoted to health care according to the degree of reader interest;

• Identification of leading information trends in the health care domain and assessment of the dynamics of their change;

• Evaluation of government policy in the field of health care according to the media coverage;

• Assessment of the dynamics of changes in the sentiment of publications on the topic of health care.

The models and the obtained results are discussed below.

3. Method

The methodological scheme of the study includes (see Figure 2) the formation of a corpus of documents, the formation of a topic model of the corpus and the identification of documents related to health care. Then the following problems are solved:

Figure 2. Rating of topics by attractiveness to readers.

• Ranking topics according to the degree of reader interest;

• Analysis of the dynamics of health care information trends;

• Evaluation of health care policy;

• Assessing the sentiment of mass media publications on health issues.

3.1. Formation of the corpus of media documents

In order to solve the stated tasks, a system of data collection from open sources of information was developed (Yakunin et al., 2021; Barakhnin et al., 2019). Using the aforementioned system, a corpus of media documents was formed (Yakunin et al., 2021; Yakunin et al., 2021). The collection of publications began in February 2020 and continues to the present time. At the time of writing the corpus consists of 7 367 372 documents related to the leading news publications of Kazakhstan and some Russian sources. The corpus includes articles from more than 20 major Kazakhstani news re-sources (Tengri News, Forbes.kz, Vlast, Komsomolskaya Pravda, and some major social networks). (see Figure 3).

Figure 3. A corpus of media documents. The percentage of content and number of articles in the corpus in thousands are given for each source.

The resulting corpus is used in the subsequent stages of analysis. The core of the analysis process is topic modeling.

3.2. Formation of the topic model of the corpus of media texts and identification of the documents related to health care

As indicated above, one of the most effective methods for analyzing the unmarked corpus texts is topic modeling. Most often researchers use the so-called latent Dirichlet allocation (LDA) (Jelodar et al., 2018) to form a topic model. In this study we used Additive Regularization on Topic Models (ARTM) (Vorontsov et al., 2015), which is an extension of LDA, the difference of which is the use of configurable regularizers that allow fine-tuning the desired result of the model: including reducing/increasing the propensity of the model to include a word and/or document in several topics simultaneously, changing the model propensity to have more/less non-zero weights in the resulting matrix, etc.

LDA can be expressed by the following equation (Blei et al., 2003): (1) $$(w,d)=\sum _{t\in T}p\left(w|t,d\right)p\left(t\right|d)=\sum _{t\in T}p\left(w|t\right)p\left(t\right|d)=\sum _{t\in T}{\phi }_{\mathit{\text{wt}}}{\theta }_{\mathit{\text{td}}}$$(1) where this sum encapsulates the aggregation of conditional distributions across all topics within the set T, where p(w|t) is the conditional distribution of words in topics, and p(t|d) is the conditional distribution of topics in documents. These ratios are valid under the assumption that there is no need to maintain the order of documents in the corpus and the order of words in the documents. The LDA method assumes that the components ${\phi }_{\mathit{\text{wt}}}$ and ${\theta }_{\mathit{\text{td}}}$ are generated by a continuous multidimensional distribution of Dirichlet probabilities. The goal of the algorithm is to find the parameters ${\phi }_{\mathit{\text{wt}}}$ and ${\theta }_{\mathit{\text{td}}}$ by maximizing the likelihood function with the corresponding regularization: (2) $$\sum _{d\in D}\sum _{\mathrm{ }w\in D}{n}_{\mathit{\text{dw}}}\mathit{\text{ln}}\mathrm{ }\sum _{t\in T}{\phi }_{\mathit{\text{wt}}}{\theta }_{\mathit{\text{td}}}+R(\phi ,\theta )\to \mathrm{\text{max}}$$(2) where $n_{\mathit{\text{dw}}}$ is the number of occurrences of word w in document; d ${\phi }_{\mathit{\text{wt}}}$ is the distribution of word w in topic t, ${\theta }_{\mathit{\text{td}}}$ is the distribution of topic t over documents d.

The additive regularization, which recovers the original word distribution over documents D by maximizing the logarithm of likelihood, is combined with the weighted sum of regularizers (3), which is based on a variety of criteria: (3) $$R\left(\phi ,\theta \right)=\sum _{i=1}{\tau }_i{R}_i\left(\phi ,\theta \right)$$(3) ${\sum }_{i=1}{\tau }_i{R}_i(\phi ,\theta )$ is a weighted linear combination of regularizers with non-negative weights ${\tau }_i.$ ARTM provides a set of regularizers implemented on the basis of Kullback-Leilbler divergence, in this case demonstrating the entropy differences between the distributions of the initial matrix p‘(w|d) and the model p‘(w|d), Dirichlet with hyperparameters ${\beta }_0{\beta }_t$ and ${\alpha }_0{\mathrm{\alpha }}_{\mathrm{t}}$ (identical to the implementation of the model of latent placement LDA, in which hyperparameters can be only positive): (4) $$R(\phi ,\theta )={\beta }_0\sum _{t\in T}\sum _{w\in W}{\beta }_{\mathit{\text{wt}}}\mathit{\text{ln}}{\phi }_{\mathit{\text{wt}}}+{\alpha }_{0 }\sum _{d\in D}\sum _{w\in W}{\alpha }_{\mathit{\text{td}}}\mathit{\text{ln}}{\theta }_{\mathit{\text{td}}}\to \mathrm{\text{max}}$$(4)

Therefore, we can identify the background topics by determining the vocabulary of the language, or calculate the overall vocabulary in the section of each document.

Decreasing regularizer, inverse smoothing regularizer are determined as follows: (5) $$(\phi ,\theta )={-\beta }_0\sum _{t\in T}\sum _{w\in W}{\beta }_{\mathit{\text{wt}}}\mathit{\text{ln}}{\phi }_{\mathit{\text{wt}}}-{\alpha }_0\sum _{d\in D}\sum _{w\in W}{\alpha }_{\mathit{\text{td}}}\mathit{\text{ln}}{\theta }_{\mathit{\text{td}}}\to \mathrm{\text{max}}$$(5) aimed at identifying significant topic words, so-called lexical cores, in addition to thematic topics in each document, zeroing out small probabilities.

De-correcting regularizer makes topics more ‘different’. Selection of topics allows the model to discard small, non-informative, duplicative and dependent topics. (6) $$(\phi ,\theta )=-0.5\mathrm{*}\tau \sum _{t\in T}\sum _{s\in \frac{T}{t}}\mathit{\text{cov}}\left({\phi }_t{\phi }_s\right)\to \mathrm{\text{max}}, \mathit{\text{cov}}\left({\phi }_t{\phi }_s\right)=\sum _{w\in W}{\phi }_{\mathit{\text{wt}}}$$(6)

This regularizer does not depend on the matrix θ. The determination of variances in the discrete distributions is carried out through ${\phi }_{\mathit{\text{wt}}}=p(w|t),$ in which the measure is the covariance of the current distribution of words in topics ${\phi }_t$ compared to calculated distributions ${\phi }_s,\mathrm{\text{where}}\mathrm{ }s\in T/t.$

The following perplexity metrics (Krasnov & Sen, 2019) were used to assess the quality of topic models: (7) $$\mathit{\text{PP}}\left(p\right)=2^{H\left(p\right)}{=2}^{-{\sum }_{x}p\left(x\right){\mathrm{\text{log}}}_{2}p\left(x\right)}=\prod _x{p\left(x\right)}^{-p\left(x\right)}$$(7) where H(p) is the information entropy of the distribution, and x is an iterator over samples (documents). The value of the perplexity metric does not have a minimum value; therefore, it is usually used to compare different models on the same dataset or to detect the ‘elbow effect’ to determine the optimal number of topics. The lower the perplexity, the better the texts from the topic group match each other.

Contrast is calculated as follows: (8) $$С\mathit{\text{on}}=\frac{1}{\left|W_t\right|}\sum _{w\in w_t}p\left(t\right|w)$$(8) where $w_t$ is the core of the topic, i.e. words from the topic whose relative weight is greater than or equal to the specified threshold. This metric allows estimating how the topics differ from each other. The higher its value is, the more contrasting the topic under the study looks against the background of the others.

Purity is determined as follows: (9) $$\mathit{\text{Pur}}=\sum _{w\in w_t}p\left(w\right|t)$$(9) where $w_t$ is also the core of the topic. Purity shows how many ‘superfluous’ words are used in a topic group. The higher the Purity value is, the fewer the common words are used in the documents related to the topic group.

The formation of a topic model describing health care was proceeded as follows. Using the BigARTM library (Vorontsov et al., 2015), a topic model consisting of 200 topic groups (topics) was formed, from which the experts selected 12 topic groups related to medicine; the experts used the mentioned Perplexity, Contrast, and Purity metrics. The topic modeling was again performed on these 12 topic groups, forming 150 next-level topic groups. 47 most relevant topics were selected from the obtained 150 topics, using the threshold for the membership matrix above 0.05. The latter were used to form the final model of 100 topic groups on a sub-corpus of 100 481 documents. Each topic group contains texts with membership greater than 0.1. The final topic model with the most relevant words for each cluster is shown in Figure 4 (Yakunin et al., 2022). In this figure we have shown some of the first and last clusters separated by a red line. The full graphical version of this model is presented in Appendix A.

Figure 4. The topic groups of texts ranked according to the degree of weight in the sub-corpus of texts devoted to health care.

The constructed topic model was used to solve the above listed problems. The results are discussed below.

4. Results and discussion

4.1. Classification of topic groups according to the degree of attractiveness for readers

In order to determine the reader interest in the topic group of publications, the number of news views was determined, which was then normalized in the range from 0 to 1. For each of the 100 final topic groups (topics), the average value of this normalized indicator (topic attractiveness TA) was calculated. For each topic, the main words, volume (number of documents included in it), weight (calculated according to (Mukhamediev et al., 2020)), TA and headings of the articles most relevant to the topic are determined.

The results show that TA ranges from 0.13 to 0.69. The lowest TA indicator was recorded for the topic described by the words: ‘kazinform, interesting, mia, mia_kazinform, correspondent_mia, correspondent_mia_kazinform, transfer_correspondent_mia, trans-fer, kazinform_link, transfer_mia_kazinform’. Relevant article titles: ‘Kazakh athletes are not sent on business trips to the countries with coronavirus’, ‘Deputy Prime Minister of Uzbekistan dies’, ‘First case of coronavirus detected in Uzbekistan’. Obviously, during the period under consideration, the respondents were practically not interested in sports events and foreign news in the context of health care. At the same time, high TA was rec-orded in the topic group described by the words ‘region, Kazakhstan_region, Kazakhstan, Almaty, city, Atyrau, Turkestan, Shymkent, Zhambyl, Atyrau_region…’. The title of the most relevant article: ‘The number of COVID-19 cases exceed 165 thousand in Kazakhstan’. This indicator reflects the fact that during the pandemic, respondents with increased attention read the news about the spread of the epidemic in the regions of Kazakhstan and the country as a whole. Topic clusters, ranked by degree of TA, are shown in Figure 5. In this figure we have shown some of the first and last clusters separated by a red line. The full version of the figure is presented in Appendix A.

Figure 5. Topic groups of media articles on health care, ranked by topic attractiveness.

4.2. Health care information trends

Considering that the coronavirus pandemic had an active impact on the information field in the period in question (2020–2022), significant topic groups were compared with the objective indicators of the pandemic (Dong et al., 2020), such as:

• The total number of new coronavirus tests performed daily;

• Percentage of positive test results as average over 7 days (inverse of tests per case);

• New cases smoothed 7-day value of new confirmed cases of coronavirus;

• 7-day smoothed rate of new coronavirus related deaths;

• 7-day average number of tests performed per confirmed case of coronavirus, which is the inverse of the positivity rate;

• Real-time assessment of the productive multiplication rate (R) of the coronavirus.

Stringency index (Stringency Index of Governmental Response: this indicator combines 9 response indicators such as school and workplace closures, travel bans, etc.; the value is scaled from 0 to 100 (100 is the most stringent response)).

The research (Yakunin et al., 2021) assessed the pairwise correlation between the size of the topic group and the COVID-19 epidemiological indicators listed above. At the same time, the topic group that best correlates with the dynamics of changes in the pandemic indicator was selected to assess the dynamics.

It was found that in 2020 the topic of the remote learning for schoolchildren was actively raised only with the beginning of the academic year (in September), and there is no direct correlation with the smoothed number of new cases. However, in 2021 – 2022 there is a strong correlation with the smoothed number of new cases. It can be inferred that the public anticipated the easing of quarantine restrictions and a return to in-person education; however, a deteriorating epidemiological landscape heightened interest in these developments. The topic of fakes in the field of health care continues to be relevant, and unlike the topic of vaccination, medicines and the general epidemiological situation, publication activity does not fade. In early 2022, the link between publication output and epidemiological indicators weakened relative to the first half of 2021. The association with search queries regarding the coverage of the economic crisis, remote work, microcredit, and similar topics has intensified. The increase in correlation with the Stringency index is especially noticeable, especially in connection with the dramatic changes (removal) of quarantine restrictions. At the same time, the correlation with issues related to health care, vaccinations, etc. decreased. This may indicate that the population is more concerned about the pragmatic issues associated with the quarantine restrictions than about health issues. Correlations with relative indicators, such as reproduction rate and single case tests, increased. These indicators reflect the epidemiological situation more objectively than the absolute indicators. This is an important indicator that the media began to reflect the epidemiological situation in the country more objectively in the end of 2021 compared to the initial period of the pandemic.

By the end of 2022, there were additional changes in the size of the mentioned topics (Figures 6–11).

Figure 6. Dynamics of the publication activity on the topic ‘Incidence, School, Child, Growth, Epidemiological’.

Figure 7. Dynamics of the publication activity on the topic ‘Vaccination, Vaccines, COVID”.

Figure 8. Dynamics of the publication activity on the topic ‘Remedy, Medicinal, Drug, Medicine, Medicinal_Remedy’.

Figure 9. Dynamics of the publication activity on the topic ‘Fake, False Information, Disinformation’.

Figure 10. Dynamics of the publication activity on the topic ‘Crisis, Lending, Debt, Microcredit’.

Figure 11. Dynamics of the publication activity on the topic ‘Case, Register, Coronavirus, Infec-tion, Coronavirus_Infection’.

The graph in Figure 11 shows that the interest in pandemic virus COVID-19 is steadily declining. So, if we compare January 2021 and January 2023, interest in this topic has dropped by about an order of magnitude. The value on the ordinate axis is the ratio of the sum of weights of documents in this topic to the sum of all weights of all topics for a given period. That is, the value can be interpreted as the share of the given topic in the information flow. It can be seen that at the peak of interest at the beginning of 2020, one topic related to pandemicCOVID-19 occupied about 10% of all information in the mass media. If we take into account all pandemic COVID-19-related topics, then the total share of media information related to the pandemic reached a peak of 15%. At the beginning of 2023, this figure fell to 1%. The one percent figure is significant, but quite comparable to other topics. For example, the topic of artificial intelligence, estimated in this way, is between 1 and 5 percent.

On the whole, the graphs above show that the volume of publications in the considered topic groups falls approximately from the middle of 2022. The exceptions are the topic groups described by the groups of words ‘crisis, lending, debt, microcredit’, ‘fake, false information, disinformation’.

4.3. Assessing coverage of health policy

The issue of assessing the media coverage of health care policy has become especially relevant over the past three years also in relation to the pandemic. The media has the potential to influence the health policy. For example, an integrated systematic review (Shen et al., 2021) presents an assessment of the impact of planned media interventions, including social networks, on the health policy-making process.

Using the above-described corpus of media texts and the Health Code (On Public Health and Healthcare System, n.d), an inter-corpus topic imbalance analysis was conducted to understand the level of public involvement in policy making and the most interesting health topics covered by the media. To assess the measure of influence of the information, it is proposed to use information on the distribution of documents by corpora within each topic, that the document belongs to a given topic (Figure 12).

Figure 12. Imbalance across the corpus of news and health code.

The figure shows the imbalance in the news corpus in dark blue, and the imbalance in the health code in light blue; the result of about 0.5 shows the topics of health policy that are particularly covered in the media. These include:

• Fund, medical, tenge, service, social;

• Fact, court, violation, article, case;

• Disease, Senior, Reaction, Vaccination;

• Driver, movement, fetus, week, term;

• Body, relative, deceased, TV channel, quarantine;

• Testing, ENT (single national testing), national, education, passing;

• Cigarette, product, tobacco;

• Development, develop, scientific, technology, production.

These are precisely the topic groups of media texts where the discussion of the actions of the government and its strategic documents on health care is most fully developed.

4.4. Assessment of the sentiment of media publications on health issues

The topic model made it possible to exclude the personal bias from the analysis process, which increased the usefulness of the model in the task of assessing the reflection of the epidemiological situation in the media. At the same time, it was found that this approach has two main limitations: taking into account the dynamic weight of only one topic and the impossibility of taking into account the opinion of experts. To eliminate these limitations, the MMA (Mass Media Assessment) method described in (Mukhamediev et al., 2020) was used. This method based on topic modeling and expert labelling of topics by sentiment. It allowed the creation of classification models with low volume high-level manual labeling. Instead labelling each document, expert labels topics by sentiment. In this case we are talking about labeling the topic groups by mood in the range from −1 to +1. The result for each document was obtained by a summation of expert labeling results weighted by document related to each topic. The mass media score is obtained by summing up the scores of documents related to the media source and subsequent normalization. In some cases, this approach made it possible to achieve an ROC AUC of 0.93, which is comparable to modern deep learning classifiers (Yakunin et al., 2021). Using the above mentioned method, there was formed the distribution of the media for the queries, which are ‘maximum negative’ and ‘maximum positive’. The results of topic modeling and sentiment analysis according to the criterion of ‘maxi-mum negative’ and ‘maximum positive’ are shown in Figures 13 and 14.

Figure 13. Imbalance across the corpus of news and health code.

Figure 14. Negative and positive media on the topic of health care, January 2023. The articles of a positive sentiment are shown in green, negative in red, and articles of a neutral sentiment are in yellow in the total volume of media articles.

It should be noted that the definition of sentiment differed from the generally accepted one. We defined the sentiment not as the author’s opinion on any issue, but as the general positive or negative of the described event for society.

5. Conclusion

Topic modeling allows evaluation of large groups of textual documents. The authors of this research solve a set of problems that evaluate the representation of health care in the media, applying topic modeling and the method described in (Mukhamediev et al., 2020). The method of evaluating such information image may be different. After selecting the documents related to health care, we chose 4 assessment methods.

First, we evaluated the topic groups of documents according to the degree of reader interest, which made it possible to identify the most relevant topics in the time period under consideration. In our case, a fairly predictable result was obtained: topics related to the spread of coronavirus significantly exceeded sports medicine topics in terms of interest during the pandemic.

Second, we analyzed the main information trends available in articles on health care and assessed the correlation of trends with objective indicators of the coronavirus pandemic. This result can be used with certain reservations to assess the objectivity of the media: the higher the degree of such correlation is, the more objective way how the media de-scribe information trends during the pandemic.

Third, by comparing state government and media documents, we assessed the extent to which the state policy in the field of health care is reflected in the period under review and identified those topic groups where it is most fully represented. These groups include documents of legal, economic and educational nature.

Fourth, we assessed the sentiment of health publications in aggregate and by individual mainstream media. We identified clear trends of the increase in the number of negative sentiment documents during the periods of increased incidence of COVID-19, as well as an increase in positive sentiment articles over time, which is associated both with the publication policy and the gradual improvement in the quality of treatment.

The applied method, based on topic modeling, proved to be a fairly effective mechanism for assessing the publication activity of the media on the topic of health care. It al-lowed:

• Ranking health related topics according to the degree of reader interest;

• Identifying health care information trends;

• Identifying those topic groups in which the state policy in the field of health care is discussed;

• Assessing the dynamics of changes in the publication activity of the mass media on health issues.

The considered four-sided ‘portrait’ can be used as an indicator of infodemic. The sharp increase in negative articles and the increase in the number of publications in the topic ‘fake, false information, disinformation’ can be considered as evidence of an infodemic that can be quantified.

6. Limitations of the study

Despite the merits of the applied method, it also has significant limitations associated with the use of a statistical language model:

1. The method does not allow to for the identification of complex semantic contexts, since the model operates with statistical patterns, and not with semantic patterns that can be obtained using the vector representation of words and texts;

2. In this regard, the model allows evaluating only general patterns, but is not able to identify subtle differences, such as: sarcasm, irony, humor, etc.

7. Future research

We assume, it will be useful to apply the embedded topic model (Dieng et al., 2020) in a future study to solve the designated problems. In addition, it seems important to explore the ‘portrait’ of health care in a relatively calm period of time free of pandemics.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Correction Statement

This article has been corrected with minor changes. These changes do not impact the academic content of the article.

Additional information

Funding

This research was funded by the Science Committee of the Ministry of Education and Science of the Republic of Kazakhstan, Grant No. AP09259587, ‘Developing of methods and algorithms of intelligent GIS for multi-criteria analysis of healthcare data’ and Grant No. BR21881908, Complex of urban ecological support (CUES). This investigation develops results of the project ‘Development of methods of healthcare system risk and reliability evaluation under coronavirus outbreak’ which has been supported by the Slovak Research and Development Agency under Grant no. PP COVID-20-0013. The work was partially supported by the Integrated Infrastructure Operational Program for the project: Systemic Public Research Infrastructure - Biobank for Cancer and Rare diseases, ITMS: 313011AFG5, co-financed by the European Regional Development Fund.

Appendix A

The topic groups of texts ranked according to the degree of weight in the sub-corpus of texts devoted to health care represented in the Figure A1.

Figure A1. The final topic model with the most relevant words for each cluster.

Figure A2. Topic groups of media articles on health care, ranked by topic attractiveness.