Enhancing Research Support for Humanities PhD Teachers: A Novel Model Combining BERT and Reinforcement Learning

In recent years, the application of artificial intelligence (AI) in research management has attracted increasing attention, as institutions look for more efficient ways to optimize resources and improve academic productivity[9]. AI technologies, especially deep learning and machine learning, have proven to be powerful tools in enhancing resource management by identifying key trends and patterns in large datasets that are often difficult to detect using traditional methods[25]; [31]. AI-driven approaches allow universities to make more informed decisions about resource allocation, directing funds and support to areas with the most potential for impactful research.

Additionally, AI can automate various time-consuming administrative processes, including grant application management, faculty performance evaluation, and project review, which traditionally require significant manual effort[21]. Automating these tasks frees up valuable resources and allows institutions to focus on strategic decision-making. Furthermore, AI technologies provide institutions with real-time insights into research output trends, enabling timely adjustments to resource allocation strategies as new data becomes available[16].

Several recent studies have highlighted the potential for AI to transform how universities approach long-term research planning. For example, AI can analyze research portfolios, funding trends, and collaboration networks to offer institutions predictive insights into areas of emerging scientific interest[17]. This allows for more dynamic and flexible management strategies, ultimately improving research efficiency. Moreover, AI’s ability to adapt to new patterns in the data ensures that universities remain responsive to shifting academic landscapes, thus improving their ability to compete in global academic rankings[7].

By implementing AI-driven systems, universities can not only optimize the allocation of limited resources but also improve collaboration opportunities and foster interdisciplinary projects[20]. AI models enable a better understanding of resource utilization across departments, ensuring that the institution’s strategic objectives are aligned with actual performance. This increasing integration of AI in research management signals a shift toward data-driven decision-making and a more efficient academic ecosystem.

Accurately predicting research output is crucial for effective research management, particularly in higher education where resources are limited and need to be utilized efficiently. AI technologies, such as machine learning, have been increasingly employed to build predictive models capable of forecasting academic productivity based on historical data[1]. These models allow institutions to anticipate future research outcomes, enabling them to allocate resources in ways that maximize impact. Such predictive approaches are critical for strategic planning, particularly in large institutions where resource allocation decisions need to account for varying research priorities and goals[8].

The adoption of predictive models enables universities to move beyond traditional research evaluation metrics such as citation counts, expanding to more complex indicators like collaboration effectiveness, research visibility, and interdisciplinary reach[5]. These more comprehensive assessments allow institutions to align their resource distribution strategies with long-term objectives, ensuring that funding and support are directed to areas with the greatest potential for academic excellence[2]. Additionally, AI-driven predictive models provide early warnings about potential research bottlenecks or inefficiencies, enabling institutions to address challenges before they significantly impact productivity[32].

Recent studies have demonstrated the usefulness of AI in predicting the success of research projects based on variables such as team composition, prior publication records, and access to research facilities[10]. By integrating these predictive capabilities, institutions can reduce the risks associated with funding allocation and support projects that are most likely to generate impactful results. Predictive models also help institutions manage their research portfolios more effectively, allowing them to track ongoing projects and make adjustments as needed to maximize output[28].

Moreover, AI-based predictive models have the potential to improve interdisciplinary research by identifying connections between seemingly unrelated fields. This fosters collaboration and opens new avenues for innovation[19]. As AI technologies continue to advance, predictive models are expected to play an even more central role in helping institutions manage their research activities, improve output quality, and enhance overall academic performance.

In our framework, the BERT (Bidirectional Encoder Representations from Transformers) model is used to process the research text data of humanities PhD teachers, such as research papers and project reports. BERT has strong contextual understanding capabilities, enabling the system to extract rich semantic information from unstructured research data. The detailed process of how BERT processes the text and generates context-aware embeddings is illustrated in Figure 2.

Figure 2.

BERT Input Representation Structure. This figure shows the BERT model’s input representation, combining token, sentence, and positional embeddings to capture contextual and sequential information.

First, the raw text dataT = {w₁, w₂, …, w_n} is tokenized into a sequence of words w_i, where each word is mapped to an initial embedding vector v_wi in a high-dimensional space using an embedding function f_embed: (1) VWi=fembed(wi) where v_wi. represents the semantic feature of word w_i. To capture positional information in the sequence, a positional encoding p_i is added to the embedding: (2) VWi′=vwi+pi where p_i ensures that the position of each word in the sequence is represented, preserving the order of the text.

Next,vwi′ is passed through multiple layers of bidirectional Transformers, where each layer applies a self-attention mechanism to compute the relationship between words in the sequence. For word w_i, the query Q_i, key K_j, and value V_j vectors are defined as: (3) Qi=Wqvwi′,Kj=Wkvwj′,Vj=Wvvwj′ where W_q, W_k, W_v are learnable weight matrices. Using these, the attention weight a_i j between word w_i and w_j is calculated as: (4) aij=softmax((QiKj⊤)dk) where d_k is the dimension of the key vectors, and the softmax ensures normalization of attention weights across all words.

The contextual embedding h_wi for word w_i is computed as the weighted sum of value vectors: (5) hwi=∑j=1naijVj where h_wi incorporates information from all other words in the sequence weighted by their attention scores.

Each Transformer layer further processes these embeddings using a feed-forward network (FFN) and layer normalization. The transformation is given by: (6) hwi(new)=LayerNorm(FFN(hwi)+hwi) where the residual connection h_wi + FFN (h_wi) improves training stability, and LayerNorm stabilizes the outputs.

After L layers, the final context-aware embedding for each word W_i is: (7) ewi=hwi(L)

These embeddings, E = {e_w1, e_w2, …, e_wn}, represent the semantic relationships between words within the context of the entire text.

These context-aware embeddings are then used as input to the Graph Neural Network (GNN) module to further analyze the dependencies within the research activities. By generating these semantic embeddings, BERT enables the model to understand deeper meanings in the text, helping the system to identify potential bottlenecks in the research process. These embeddings improve the interpretability of the text data and provide a strong foundation for accurate predictions. In summary, BERT is primarily used in this network to transform complex textual data into meaningful vector representations, enabling the combination of text information with structured data (such as research collaboration networks), laying the groundwork for further analysis and prediction.

In our proposed framework, the Graph Neural Network (GNN) is designed to model the intricate relationships between research activities, such as collaborations, citations, and project interactions. By leveraging both the text embeddings generated by BERT and the structured data from research activities, the GNN allows for a detailed analysis of the complex connections within the academic network.

Humanities PhD teachers’ research activities can be represented as a graph structure G = (V, E),where V denotes the set of nodes, each corresponding to a specific research activity (e.g., a paper, project), and E represents the edges, which capture the relationships between these activities (such as collaborations or citations).Each node v has an associated feature vector X_v, which includes both the BERT-generated embeddings and structured data like the number of citations or collaborations.

The GNN updates node features by aggregating information from neighboring nodes. The node update equation is defined as: (8) hv(l+1)=σ(W(l)·AGGREGATE({ hu(l):u∈N(v) })+b(l)) where hv(l) is the feature vector of node v at layer l, N(v) denotes the set of neighboring nodes, W^(l) is the weight matrix at layer l, b^(l) is the bias term, and σ is the activation function. This iterative update enables each node to capture both direct and indirect dependencies in the graph. Message passing, a key component of the GNN, computes interactions between nodes to aggregate information. The message aggregation is defined as: (9) mv(l)=∑u∈N(v)ϕ(hv(l),hu(l)) where mv(l) is the aggregated message for node v at layer l ,and φ is the message function that computes the interaction between nodes v and u. The aggregated information is combined with the node’s own feature to generate updated node features: (10) hv(l+1)=ψ(hv(l),mv(l)) where ψ is an update function (e.g, concatenation followed by a linear transformation) that integrates the original node features with the aggregated message.

To ensure stability and proper scaling during updates, the node features are normalized after each layer: (11) hv(l+1)=hv(l+1)‖hv(l+1)‖2 where ‖hv(l+1)‖2 denotes the L2 norm of the feature vector hv(l+1) The final representation of each node after L layers is obtained by combining information from all layers: (12) zv=CONCAT(hv(l),hv(2),…,hv(L)) where z_v is the final feature vector used for downstream tasks, and CONCAT denotes the concatenation operation.

The architecture of the GNN within our framework is illustrated in Figure 3. This architecture allows the model to gain a comprehensive understanding of how collaborations, citations, and other relationships influence the overall research network and helps in identifying key bottlenecks in the research process.

Figure 3.

Structure of the Recurrent Neural Network

To ensure optimal distribution of research resources, we integrate a Reinforcement Learning (RL) module into our framework. This module dynamically learns the best strategies for allocating limited resources, such as funding and support, to maximize research output. The RL component continuously adapts to changes in research needs and constraints, making it a crucial part of the model’s decision-making process.

The resource allocation problem is modeled as a Markov Decision Process (MDP). The MDP is defined by a tuple (S, A, P, R), where Srepresents the state space, describing the current allocation of resources to various research activities; A is the action space, which contains all possible resource allocation strategies (e.g, assigning different levels of funding or support to various researchers); P(s′|s,a) is the state transition probability, defining the likelihood of moving from state s to state s′ after taking action a; and R(s, a) is the reward function, which assigns a numerical value based on how effective the action a is in improving research output. At each decision point, the RL agent observes the current state s ,selects an action a ,and receives a reward R(s, a) based on the effectiveness of the resource allocation. The objective of the RL agent is to maximize the cumulative future rewards G_t, which is defined as: (13) Gt=∑k=0∞γk Rt+k where γ is the discount factor, which balances the importance of immediate versus future rewards. The RL agent must learn an optimal policy π(a|s), which selects the action a that maximizes the expected long-term reward given the current state s.

To achieve this, we employ the Q-learning algorithm to iteratively update the value function Q(s, a),which represents the expected cumulative reward for taking action a in state s.The Q-value update rule is given by: (14) Q(s,a)=Q(s,a)+α(R(s,a)+γmaxa'Q(s',a')−Q(s,a)) where α is the learning rate, and max_a′ Q(s′, a′) represents the maximum expected future reward for the next state s′. Over time, this iterative update process ensures that the agent improves its policy π(a|s).

To balance exploration and exploitation, the agent follows an epsilon-greedy policy.

The action selection rule is: (15) at={argmaxaQ(s,a)with probability 1-ϵrandom action from A with probability ϵ where ∈ controls the exploration rate, decaying over time to allow more exploitation as the policy improves.

The policy π aims to maximize the expected return G(s), defined as: (16) G(s)=Eπ[∑t=0∞γtrt∣s0=s] where Eπ is the expected value under the policy π.

Finally, to stabilize training and ensure convergence, we include a reward normalization step: (17) r^t=rt−μrσr where μ_r and σ_r are the mean and standard deviation of rewards observed so far. This normalization improves the agent’s ability to handle varying reward scales effectively

By continuously updating the Q-values and refining its policy, the RL agent learns to allocate resources optimally, maximizing both short-term and long-term research productivity. The system dynamically adjusts resource allocations in response to evolving research needs and outcomes, ensuring that resources are used efficiently and that research output is maximized.

We implement our model using Python and deep learning frameworks such as PyTorch and TensorFlow. The system is trained on a server with an NVIDIA GPU, ensuring efficient computation during the model training phase. The training process is divided into two main stages: (1) the initial training of the BERT and GNN modules for research activity modeling, and (2) reinforcement learning for optimizing resource allocation strategies based on the feedback from predicted research outcomes.

The hyperparameters for BERT, GNN, and reinforcement learning are tuned to ensure optimal performance. Specifically, BERT uses a pre-trained model with a learning rate of 2e- 5, while the GNN employs three layers with ReLU activation. The reinforcement learning module uses a Q- learning algorithm with a learning rate α = 0.01, and the discount factor γ = 0.9 to balance short-term and long-term rewards. The epsilon e for the epsilon-greedy policy is set to decay from 1.0 to 0.1 over the training period to ensure sufficient exploration.

To validate the effectiveness of our model, we selected two widely-used and high-quality academic datasets: the Semantic Scholar Open Research Corpus (S2ORC)[20] and the Microsoft Academic Graph (MAG)[27]. These datasets provide a combination of textual and structured data, enabling comprehensive support for the various modules of our model.

S2ORC is a large-scale open-access academic document dataset, spanning multiple academic fields and well-suited for natural language processing and text mining tasks. The dataset contains approximately 81 million academic articles, each accompanied by metadata such as author information, publication year, publication type, citation records, and subject tags. In our research, S2ORC serves as the source of textual data for training the BERT model. By extracting text data from academic papers, abstracts, and research reports, BERT can generate context-aware semantic embeddings, which serve as a strong semantic foundation for the subsequent Graph Neural Network (GNN) module. Additionally, S2ORC’s citation information provides valuable input for modeling relationships between research activities.

MAG is a large-scale academic graph dataset that provides structured data, including information on academic papers, authors, institutions, citation relationships, and collaboration networks. MAG is ideal for analyzing academic collaboration networks and citation patterns. In our experiments, MAG is primarily used to construct research collaboration and citation networks, which are fed into the GNN and reinforcement learning modules for academic relationship modeling and resource allocation optimization. By extracting collaboration data from MAG, we can capture patterns of cooperation among researchers, and this data is further used to optimize decisions on research resource allocation.

By combining the S2ORC and MAG datasets, our framework effectively leverages both textual and structured data to analyze research activities, predict research output, and optimize resource allocation, ensuring that the model has sufficient expressiveness and generalizability across different levels of academic research tasks.

In this study, we employed several key evaluation metrics to assess the capability of our deep learning model in identifying and analyzing the research difficulties faced by PhD teachers in the arts. Initially, topic coherence metrics help us evaluate the quality and relevance of the themes extracted from the text by the model. Precision and recall are used to measure the model′ s accuracy in correctly identifying teachers facing research difficulties, while the F1 score, as the harmonic mean of precision and recall, provides a comprehensive indicator of the model′s overall performance. Additionally, model stability is assessed through repeated tests on different data subsets to ensure consistency in model outputs. The effectiveness of the network structure is verified through the analysis of the teacher collaboration network, ensuring that our approach accurately reflects the patterns of collaboration and research activity within the network. These evaluation metrics collectively reflect the model′s capability in academic applications and its effectiveness in practical implementation.

In Table 1, we present a comparative analysis of the performance of several deep learning models applied to the PAMAP2 and MHEALTH datasets, assessed using four key metrics:

The comparative analysis of the models presented in the table 1 above reflects their performance on two distinct academic datasets: S2ORC and MAG. The model denoted as “Ours,” demonstrates the highest levels of precision, recall, and F1 scores across both datasets, underscoring its efficacy in addressing the nuanced challenges of identifying research difficulties among PhD teachers in arts disciplines. On the S2ORC dataset, “Ours” achieved a precision of 0.85 and an F1 score of 0.87, slightly better than its performance on the MAG dataset, where it scored 0.87 in precision and 0.88 in F1. This indicates a consistent yet slightly varied ability to generalize across different academic texts, likely due to the diverse nature and scope of articles within each dataset. The MAG dataset, known for its broader interdisciplinary articles, may provide a more complex challenge in identifying specific research difficulties, which could explain the minor performance variation. Furthermore, the comparison with other advanced models like Transformer + DNN, BERT + LSTM, and GPT-3 highlights the added value of our approach, especially in the context of granular and context-specific tasks such as evaluating academic difficulties. While models like GPT-3 perform commendably with precision scores up to 0.85 on MAG, the tailored approach of combining BERT′ s contextual understanding with DNN′s learning capabilities and the strategic optimization of reinforcement learning in “Ours” provides a more focused analysis on the datasets tailored to the specific needs of arts PhD research. These results not only validate the robustness of “Ours” in handling varied data characteristics but also illustrate the potential for specialized models in academic analytics, where understanding the context and nuances of academic challenges is crucial. This analysis suggests that our model could be particularly useful for educational institutions aiming to better support their doctoral candidates by pinpointing and addressing specific research hurdles.

The results table 2 highlights the performance of our proposed model in identifying key research challenges from both the S2ORC and MAG datasets. The theme of “Funding challenges” stands out, with high consistency scores of 0.82 in S2ORC and 0.85 in MAG, reflecting the model′ s ability to effectively capture this common issue across different academic contexts. Additionally, the number of related publications, 95 in S2ORC and 120 in MAG, supports the significance of this challenge, showing that it is widely discussed in the literature. Similarly, for the theme “Resource scarcity,” the model maintains consistent performance, with scores of 0.79 in S2ORC and 0.80 in MAG. This indicates that the model reliably identifies resource limitations as a key obstacle in academic research. The theme is linked to 80 publications in S2ORC and 110 in MAG, further confirming its relevance in both datasets.

The model also performs well in recognizing “Publication bias,” with high consistency scores of 0.86 in S2ORC and 0.88 in MAG. This suggests the model′ s accuracy in detecting issues related to biases in scholarly publishing, which is reflected by 65 related publications in S2ORC and 90 in MAG. Although the number of related documents is slightly lower than other themes, the high scores show that the model captures the essence of this issue effectively. “Collaboration issues,” on the other hand, have slightly lower consistency scores of 0.77 in S2ORC and 0.78 in MAG, suggesting that the model finds this theme more challenging to extract, possibly due to the more nuanced and multi-dimensional nature of collaboration problems in academic settings. The number of related publications, 55 in S2ORC and 70 in MAG, supports this interpretation, indicating that collaboration difficulties are less frequently discussed or more complex to quantify in academic literature.

Overall, our proposed model demonstrates strong consistency across both datasets in identifying key themes, with minor variations in performance that highlight the distinct characteristics of the datasets. The analysis underscores the model′ s ability to effectively extract and represent the research challenges faced by PhD scholars in arts disciplines.

This study combines BERT, RNN, and reinforcement learning methods to automatically extract and analyze the primary research difficulties faced by PhD teachers in new arts colleges, proposing targeted breakthrough strategies. These techniques effectively identify the correlation between five major research difficulties (Funding Challenges, Resource Scarcity, Publication Bias, Collaboration Issues, Methodological Issues) and their corresponding breakthrough strategies. BERT is responsible for semantic analysis of a large number of relevant documents, extracting key terms related to research difficulties. RNN is used to analyze the temporal relationship between difficulties and strategies, while reinforcement learning optimizes the strategy matching results.

In Figure 4, for Funding Challenges, the most effective strategy is Increased Funding Support. This indicates that directly increasing funding has a significant effect in alleviating the funding shortages faced by PhD teachers. Additionally, for Resource Scarcity, the primary breakthrough strategy is Building Research Networks, which effectively addresses the issue of insufficient resources by promoting cooperation and resource sharing. Regarding Publication Bias, the most effective measure is Reducing Workload, which shows that by reducing non-research tasks, teachers can focus more on academic publications, thereby mitigating the effects of publication bias. Meanwhile, Collaboration Issues can be significantly alleviated by Fostering Collaboration. This strategy highlights the importance of establishing collaborative relationships, particularly in new colleges, where interdisciplinary and inter-institutional collaboration can bring more research opportunities. Lastly, for Methodological Issues, the most effective strategy is Enhancing Methodology. Improving research methodology can help strengthen teachers′ research capabilities and overcome technical difficulties in research design. These findings are further validated through the combination of BERT, RNN, and reinforcement learning. Reinforcement learning, through iterative strategy optimization, selects the optimal strategy combinations, confirming the matching relationship between various research difficulties and corresponding breakthrough strategies.

Figure 4.

Effectiveness of breakthrough strategies in addressing the primary research difficulties faced by PhD teachers in new arts colleges.

Figure 5 presents the convergence speed of five strategies for optimizing research resource allocation: Increased Funding Support, Building Research Networks, Enhancing Methodology, Reducing Workload, and Fostering Collaboration. Each line in the graph represents the resource allocation efficiency over a series of iterations, highlighting how each strategy improves resource distribution. The graph shows that all strategies exhibit initial rapid improvement, with diminishing returns as they approach maximum efficiency. Strategies like Increased Funding Support and Enhancing Methodology converge more quickly, reaching near-optimal efficiency within the first 10 iterations, suggesting their effectiveness for short-term interventions. On the other hand, strategies such as Fostering Collaboration and Reducing Workload, while still effective, demonstrate a slower convergence. These strategies appear to require more time and iterations to reach similar levels of efficiency, indicating that they may be better suited for long-term optimization. Despite their slower convergence, the consistent upward trend suggests that these strategies continue to yield improvements, albeit at a slower pace. Overall, the graph highlights the differences in how quickly each strategy can optimize research resources, allowing for strategic decision-making depending on whether immediate or sustained improvements are desired.

Figure 5.

Convergence speed of different research resource allocation strategies.