CN120146046A - A low-resource topic key topic extraction method for online forums - Google Patents
A low-resource topic key topic extraction method for online forums Download PDFInfo
- Publication number
- CN120146046A CN120146046A CN202510615488.7A CN202510615488A CN120146046A CN 120146046 A CN120146046 A CN 120146046A CN 202510615488 A CN202510615488 A CN 202510615488A CN 120146046 A CN120146046 A CN 120146046A
- Authority
- CN
- China
- Prior art keywords
- document
- topic
- resource
- low
- enhanced
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Machine Translation (AREA)
Abstract
The application belongs to the technical field of natural language processing and text mining, and discloses a low-resource topic key subject extraction method for an online forum, which comprises the steps of carrying out data enhancement on the semantic retention of an original text through a large language model to generate an enhanced document set; extracting context-aware semantic representation of a document by using a pre-training language model, constructing a learnable topic embedding matrix, calculating to generate topic distribution, designing a semantic-aware contrast learning framework, adopting a dynamic negative sample screening strategy to optimize topic diversity, and simultaneously ensuring topic consistency by using priori alignment loss. The application innovatively fuses the LLM enhanced data expansion mechanism and the lightweight topic coding architecture, effectively solves three technical problems of data sparseness, model overfitting and noise sensitivity in a low-resource scene through the double optimization of contrast learning regularization and priori distribution matching, and provides an efficient and reliable topic modeling solution for social media public opinion analysis.
Description
Technical Field
The application belongs to the technical field of natural language processing and text mining, and particularly relates to an on-line forum low-resource topic-oriented key subject extraction method.
Background
With the popularity of the internet, the internet forum has become a popular platform for users to share daily life and communicate with opinion. These user-generated text provides valuable resources for analyzing public opinion about various social phenomena. However, most sub-forums on forums show low resource attributes, active members and limited posts. Despite the small size of the corpus, articles submitted by users contain important aspects such as "mental health and family problems," such as "autism and romantic relationships," such as "health and medical struggling," all of which can lead to a sense of discomfort and have potential analytical value.
The traditional topic modeling technology faces three major core challenges, namely a) data sparsity, namely the number of low-liveness community documents is often less than 3000, and the neural network is insufficiently trained. b) Noise sensitivity-user generated text has misspellings and irregular grammar problems, and affects the word bag model effect. c) Model complexity-neural topic model parameters of the VAE architecture are excessive, and overfitting easily occurs on small data sets.
The existing solutions have significant drawbacks, such as traditional bayesian approaches, such as LDA and BOW-based neural variants, such as ECRTM, that fail to address the challenges of data scarcity and noise information, as well as recently developed VAE-based contextualized topic models, such as the downsampled contextualized topic model CTMNeg and the contextualized word topic model CWTM, that are poor at handling the challenges of data scarcity and model complexity.
Disclosure of Invention
In order to solve the technical defects, the application provides a low-resource topic key topic extraction method for an online forum, which solves three technical problems of data sparseness, noise interference and model complexity through a LLM enhanced low-resource topic modeling framework, realizes topic consistency improvement, topic diversity improvement and calculation efficiency improvement, discovers topics with higher quality from the low-resource online forum, and is suitable for semantic analysis and public opinion insight of User Generated Content (UGC) in a social media platform.
In order to achieve the above purpose, the application is realized by the following technical scheme:
The application discloses a low-resource topic key topic extraction method for an online forum, which specifically comprises the following steps of:
Step 1, acquiring a low-resource document of an online forum, and carrying out semantic retention data enhancement on the acquired low-resource document through a large language model to generate an enhanced document set;
step 2, extracting document level representation in the enhanced document set by utilizing a pre-training language model;
step 3, constructing a learnable topic embedding matrix, and calculating document topic distribution through document-topic similarity;
Step 4, designing a semantic perception contrast learning frame, screening dynamic negative samples in the same batch of reinforced documents of the contrast learning frame, calculating contrast learning loss, optimizing a theme embedding matrix, ensuring theme consistency by using priori alignment loss, and obtaining a theme word;
and 5, expanding the subject words obtained in the step 4 by using a large language model, and inducing subject insights, so that the low-resource corpus can be better understood.
The application further improves that the step 1 obtains the low resource file of the online forum, carries out the data enhancement of the semantic retention to the obtained low resource file through a large language model, and generates an enhanced file set, which comprises the following steps:
step 1.1, constructing a document enhancement prompt template based on a large language model to generate a large language model result, wherein the document enhancement prompt template comprises semantic keeping constraint conditions:
a) The minimum semantic variation principle is that each text in the generated enhanced document set is required to be similar to the low-resource document as semantically as possible;
b) Sentence fluency optimization, namely eliminating spelling errors and irregular grammar expression;
Step 1.2, iterative generation, namely calculating the embedding similarity of a low-resource document and a large language model generation result corresponding to the low-resource document by using a pre-training language model;
Step 1.3, when Individual low resource documentsCorresponding generation resultIs below a threshold valueWhen the screening mechanism is triggered, the steps 1.1-1.2 are repeatedly executed, and the first step is regeneratedIndividual low resource documentsCorresponding generation resultWhereinFor the low-resource corpus, the embedding similarity of the highest generated result in each iteration is reserved, and the low-resource corpus is collectedObtaining a first enhanced document set after two times of data enhancementAnd second enhancing the document collectionWhereinIs the number of documents in the low-resource corpus.
The application further improves that the step 2 extracts the document level representation of the document in the enhanced corpus, and specifically comprises the following steps:
step 2.1, adopting a pre-training language model to strengthen each document Encoding to obtain word embedded set corresponding to the enhanced documentI.e. the set of words that the enhanced document contains:
Wherein, ,Is a word-embedded identification,To enhance documentsMiddle (f)The number of words to be used in a word,In the form of a transducer encoder,,To enhance documentsThe number of words in the set of words,Is the dimension of the film,For the dimension size of the hidden variable,Is the firstDocument level representation of individual words;
Step 2.2, generating a document-level embedded representation corresponding to the enhanced document :
。
The application further improves the method in that the step 3 constructs a learnable theme embedding matrix, calculates the document theme distribution through the document-theme similarity, and specifically comprises the following steps:
Step 3.1, constructing a learnable theme embedding matrix WhereinIs the number of topics;
Step 3.2, obtaining the topic distribution by the document level embedded representation and topic embedded matrix dot product calculation, and each enhanced document Corresponding topic distributionThe calculation mode of (a) is as follows:
Wherein, To enhance documentsA corresponding document-level representation of the document,In order to refer to calculating the distribution of the subject matter,And (3) for example normalization operation, independent normalization of each theme dimension is realized.
In the step 4, a semantic perception contrast learning frame is designed, dynamic negative samples are screened in the same batch of reinforced documents of the contrast learning frame, contrast learning loss is calculated, a theme embedding matrix is optimized, the theme consistency is ensured by using prior alignment loss, and a subject term is obtained, and the method specifically comprises the following steps:
step 4.1, calculating the first using the topic distribution of the enhanced document as a basis Personal enhanced document embeddingAnd (d)Personal enhanced document embeddingRelative semantic relevance score between:
Wherein, Representing the calculation of cosine similarity of the two vectors.
Step 4.2, screening and comparing dynamic negative samples in the reinforced documents of the same batch of the learning framework through the following formula:
Wherein, Super parameters that are thresholds;
Step 4.3 for Low resource documents All have low resource documentsCorresponding first enhanced documentAnd second enhancing the documentWhen training the contrast learning framework, the corpus is collected from low resourcesMedium random sampling,For batch size, take random samplesIs to enhance documents for a first batch of (a)And second batch enhanced documentsSeparately computing a first batch of enhanced document topic distributionsAnd second batch enhanced document topic distributionAnd construct positive sample pairsNegative sample pair pass within the same lotScreening, and calculating the contrast learning loss of the positive sample pair and the negative sample pair of the first batchContrast learning loss for second lot positive and negative sample pairs:
Wherein, Is a hyper-parameter that is a trade-off factor between positive and negative sample pairs,Representing temperature parameters of a document-level embedded representation, total contrast learning loss for the same batchThe method comprises the following steps:
Step 4.3, randomly sampling two batch sizes, namely Prior topic distributionAnd calculate a priori alignment loss:
Wherein, Representing a topic distribution deduced from two batches of enhanced documents,In order to be a moment order,For the number of topics currently being calculated,Is the firstThe distribution of the individual topics,Is the firstThe number of a priori distributions is chosen,Representing the mean of the d-th topic of the inferred topic distribution domain,Representing the mean of the d-th topic of the prior distribution domain,Representation ofIs the first of (2)The number of subjects to be treated,Representation ofIs the first of (2)A subject matter;
Step 4.4, total loss function :
Wherein, Is a super parameter.
The application further improves that the step 4 of designing the comparative learning framework training process of semantic perception comprises the following steps:
step T1, initializing a theme embedding matrix Configuration of dimension size including hidden variable with pre-trained language model parametersInputting the low-resource document into a large language model to generate an enhanced document set by the non-zero super parameters;
step T2, carrying out batch random extraction on the enhanced document set, inputting a contrast learning frame, and carrying out prior sampling in a Dirichlet parameter space to obtain document theme distribution of the enhanced document set;
Step T3, comprehensively evaluating the total contrast learning loss of the same batch Loss of alignment with a prioriAnd circularly executing the theme embedding matrix parameter updating operation, and stopping training when the semantic aware contrast learning framework converges.
The application further improves that the large language model expansion and induction subject seeing method of the step 5 specifically comprises the steps of giving the name of the low-resource document and the corresponding name thereof in the promptThe 10 subject words corresponding to each subject are required to be generalized for each subject to generate human-readable subject insights.
The beneficial effects of the application are as follows:
The application designs a semantic perception contrast learning framework, which is a theme model for establishing low resources by using a nerve-based method for the first time.
The application designs a semantic perception text expansion and corpus expansion scheme based on a large language model, which can solve the problem of data deficiency of a low-resource corpus.
The exploration presented by the present application incorporates semantic knowledge in the converter language model into the modeling process to address model complexity and noise information challenges.
Summarizing, the method is successfully applied to an online forum low-resource text analysis scene, a complete technical path from data enhancement to multi-level topic analysis is realized, compared with a traditional topic model, the method has obvious advantages in topic semantic consistency and fine granularity knowledge discovery, and a new solution is provided for text analysis in a low-resource environment.
Drawings
Fig. 1 is a flow chart of the present application.
FIG. 2 is a diagram of a model architecture for implementing the present application.
FIG. 3 is a comparison line graph of the four indicators of different comparison models in the present application.
Detailed Description
Embodiments of the application are disclosed in the drawings, and for purposes of explanation, numerous practical details are set forth in the following description. However, it should be understood that these practical details are not to be taken as limiting the application. That is, in some embodiments of the application, these practical details are unnecessary. Moreover, for the purpose of simplifying the drawings, some conventional structures and components are shown in the drawings in a simplified schematic manner.
As shown in fig. 1, the method for extracting the key topics of the low-resource topics for the online forum of the application specifically comprises the following steps:
Step 1, acquiring a low-resource document of an online forum, and carrying out semantic retention data enhancement on the acquired low-resource document through a large language model to generate an enhanced document set, wherein the enhanced document set is specifically as described in the part (a) in fig. 2:
step 1.1, constructing a document enhancement prompt template based on a large language model to generate a large language model result, wherein the document enhancement prompt template comprises semantic keeping constraint conditions:
a) The minimum semantic variation principle is that each text in the generated enhanced document set is required to be similar to the low-resource document as semantically as possible;
b) Sentence fluency optimization, namely eliminating spelling errors and irregular grammar expression;
The prompt specifically comprises the following steps:
。
Step 1.2, iterative generation, namely using a pre-training language model, adopting Sentence-BERT in the embodiment, and calculating the embedded similarity of a low-resource document and a large language model generation result corresponding to the low-resource document, wherein the concrete use cosine similarity for measurement;
Step 1.3, when Individual low resource documentsCorresponding generation resultIs below a threshold valueWhen the screening mechanism is triggered, the steps 1.1-1.2 are repeatedly executed, and the first step is regeneratedIndividual low resource documentsCorresponding generation resultWhereinFor a low-resource corpus set,For super parameters, the embedding similarity of the highest generated result in each iteration is reserved, and the corpus collection with low resources is obtainedObtaining a first enhanced document set after two times of data enhancementAnd second enhancing the document collectionWhereinIs the number of documents in the low-resource corpus.
Step2, extracting document level representation in the enhanced document set by utilizing a pre-training language model, and specifically comprising the following steps:
step 2.1, employing Pre-trained language model Sentence-BERT for each enhanced document Encoding to obtain word embedded set corresponding to the enhanced documentI.e. the set of words that the enhanced document contains:
Wherein, ,Is a word-embedded identification,To enhance documentsMiddle (f)The number of words to be used in a word,In the form of a transducer encoder,,To enhance documentsThe number of words in the set of words,Is the dimension of the film,For the hidden variable dimension size, 768 is set in Sentence-BERT,Is the firstDocument level representation of individual words;
Step 2.2, generating a document-level embedded representation corresponding to the enhanced document :
。
Step 3, constructing a learnable topic embedding matrix, and calculating the topic distribution of the document according to the similarity of the document and the topic, wherein the topic distribution of the document is shown in the part 2 (b) of the figure, and specifically comprises the following steps:
Step 3.1, constructing a learnable theme embedding matrix This matrix is randomly initialized and parameterized throughout the training process, whereIs the number of topics;
Step 3.2, obtaining the topic distribution by the document level embedded representation and topic embedded matrix dot product calculation, and each enhanced document Corresponding topic distributionThe calculation mode of (a) is as follows:
Wherein, To enhance documentsA corresponding document-level representation of the document,In order to refer to calculating the distribution of the subject matter,And (3) for example normalization operation, independent normalization of each theme dimension is realized.
Step 4, designing a semantic perception contrast learning framework, implementing a dynamic negative sample strategy in the same batch of enhanced documents of the contrast learning framework, calculating contrast learning loss, optimizing a theme embedding matrix, ensuring theme consistency by using priori alignment loss, and obtaining a theme word;
And 5, expanding the subject words obtained in the step 4 by using a large language model, and inducing subject insights to help better understand low-resource corpora, wherein the steps are as follows:
step 4.1, calculating the first using the topic distribution of the enhanced document as a basis Personal enhanced document embeddingAnd (d)Personal enhanced document embeddingRelative semantic relevance score between:
Wherein, Representing the calculation of cosine similarity of the two vectors.
Step 4.2, screening and comparing dynamic negative samples in the reinforced documents of the same batch of the learning framework through the following formula:
Wherein, Is a threshold hyper-parameter, so that negative samples with high semantic similarity to the original sentence are masked as false negative samples.
Step 4.3 for Low resource documentsAll have low resource documentsCorresponding first enhanced documentAnd second enhancing the documentWhen training the contrast learning framework, the corpus is collected from low resourcesMedium random sampling,For batch size, take random samplesIs to enhance documents for a first batch of (a)And second batch enhanced documentsSeparately computing a first batch of enhanced document topic distributionsAnd second batch enhanced document topic distributionAnd construct positive sample pairsNegative sample pair pass within the same lotScreening, and calculating the contrast learning loss of the positive sample pair and the negative sample pair of the first batchContrast learning loss for second lot positive and negative sample pairs:
Wherein, Is a hyper-parameter that is a trade-off factor between positive and negative sample pairs,Representing temperature parameters of a document-level embedded representation, total contrast learning loss for the same batchThe method comprises the following steps:
step 4.3 from Dirichlet ) Distributed random sampling of two batch sizes, i.ePriori topic distribution of dimensionsAnd calculate a priori alignment loss:
Wherein, Representing a topic distribution deduced from two batches of enhanced documents,In order to be a moment order,For the number of topics currently being calculated,Is the firstThe distribution of the individual topics,Is the firstThe number of a priori distributions is chosen,Representing the mean of the d-th topic of the inferred topic distribution domain,Representing the mean of the d-th topic of the prior distribution domain,Representation ofIs the first of (2)The number of subjects to be treated,Representation ofIs the first of (2)A subject matter;
Step 4.4, total loss function :
Wherein, Is a super parameter.
The method for expanding and inducing the topic of the large language model in the step 5 comprises the steps of giving the name of the low-resource document and the corresponding name in the promptThe 10 subject words corresponding to each subject are required to be generalized for each subject to generate human-readable subject insights. The large language model expansion and induction subject seeing method in the step S6 specifically comprises the following steps:
the name of the post and the corresponding name are given in the prompt The 10 subject words corresponding to the respective subjects are required to be generalized for each subject to generate human-readable subject insights, and the prompt is specifically as follows:
。
the contrast learning framework training process of semantic perception comprises the following steps:
step T1, initializing a theme embedding matrix Configuration of dimension size including hidden variable with pre-trained language model parametersInputting the low-resource document into a large language model to generate an enhanced document set by the non-zero super parameters;
step T2, carrying out batch random extraction on the enhanced document set, inputting a contrast learning frame, and carrying out prior sampling in a Dirichlet parameter space to obtain document theme distribution of the enhanced document set;
Step T3, comprehensively evaluating the total contrast learning loss of the same batch Loss of alignment with a prioriAnd circularly executing the theme embedding matrix parameter updating operation, and stopping training when the semantic aware contrast learning framework converges.
The training algorithm is as follows:
In order to verify the efficacy of the application, a sub-forum layout named 'what puzzles you' is selected to construct an experimental data set, and the system evaluation is carried out through four subject continuity indexes. The core evaluation index mean is shown in table 1.
TABLE 1
As shown in FIG. 3, the indexes of the application are CP:0.2301, CA:0.1651, NPMI:0.02136, UCI: 0.2558, which are all higher than the comparison model, wherein the highest comparison model is CP:0.0163, CA:0.1413, NPMI:0.0046, UCI: 0.3225 respectively. Three representative methods, namely a comparison model 1, a comparison model 2 and a comparison model 3 are selected as comparison references. The specific model is as follows:
The comparative model 1 is the LDA method. Comparative model 2 is the vONT method. Comparative model 3 is the LLMTopic method.
CP()、CA() NPMI (Normalized Pointwise Mutual Information) and UCI) The four indexes are public evaluation indexes for quantifying the modeling quality of the subject from the dimensionality of semantic relevance, vocabulary distribution and the like.
The present exemplary embodiment is only for explaining a feasible implementation path of the technical principle, and does not constitute a substantial limitation of the patent claims. Under the premise of strictly following the innovative boundary defined by the claims of the present invention, the related technical researchers perform parameter adaptation, architecture equivalent conversion or application scene extension and other deduction actions on the technical features disclosed in the specification, and all the deduction actions should be considered to be within the radiation range of the creative contribution of the original patent, so as to legally enjoy the patent protection efficacy.
Claims (7)
1. The low-resource topic key topic extraction method for the online forum is characterized by comprising the following steps of:
Step 1, acquiring a low-resource document of an online forum, and carrying out semantic retention data enhancement on the acquired low-resource document through a large language model to generate an enhanced document set;
step 2, extracting document level representation in the enhanced document set by utilizing a pre-training language model;
step 3, constructing a learnable topic embedding matrix, and calculating document topic distribution through document-topic similarity;
step 4, designing a semantic perception contrast learning framework, implementing a dynamic negative sample strategy in the same batch of enhanced documents of the contrast learning framework, calculating contrast learning loss, optimizing a theme embedding matrix, ensuring theme consistency by using priori alignment loss, and obtaining a theme word;
and 5, expanding the subject words obtained in the step 4 by using a large language model, and inducing subject insights, so that the low-resource corpus can be better understood.
2. The method for extracting the key topics of the low-resource topics for the online forum according to claim 1, wherein the step 1 is characterized in that the low-resource documents of the online forum are obtained, the obtained low-resource documents are subjected to semantic retention data enhancement through a large language model, and an enhanced document set is generated, and the method specifically comprises the following steps:
step 1.1, constructing a document enhancement prompt template based on a large language model to generate a large language model result, wherein the document enhancement prompt template comprises semantic keeping constraint conditions:
a) Minimum semantic variation principle, namely requiring that each text in the generated enhanced document set is semantically similar to a low-resource document;
b) Sentence fluency optimization, namely eliminating spelling errors and irregular grammar expression;
Step 1.2, iterative generation, namely calculating the embedding similarity of a low-resource document and a large language model generation result corresponding to the low-resource document by using a pre-training language model;
Step 1.3, when Individual low resource documentsCorresponding generation resultIs below a threshold valueWhen the screening mechanism is triggered, the steps 1.1-1.2 are repeatedly executed, and the first step is regeneratedIndividual low resource documentsCorresponding generation resultWhereinFor the low-resource corpus, the embedding similarity of the highest generated result in each iteration is reserved, and the low-resource corpus is collectedObtaining a first enhanced document set after two times of data enhancementAnd second enhancing the document collectionWhereinIs the number of documents in the low-resource corpus.
3. The method for extracting low-resource topic key subject matter oriented to the online forum, which is characterized in that the step 2 is used for extracting document level representation of the documents in the enhanced corpus, and specifically comprises the following steps:
step 2.1, adopting a pre-training language model to strengthen each document Encoding to obtain word embedded set corresponding to the enhanced documentI.e. the set of words that the enhanced document contains:
;
Wherein, ,Is a word-embedded identification,To enhance documentsMiddle (f)The number of words to be used in a word,In the form of a transducer encoder,,To enhance documentsThe number of words in the set of words,Is the dimension of the film,For the dimension size of the hidden variable,Is the firstDocument level representation of individual words;
Step 2.2, generating a document-level embedded representation corresponding to the enhanced document :
。
4. The method for extracting low-resource topic key topics facing to the online forum as set forth in claim 3, wherein the step 3 is to construct a learnable topic embedding matrix, calculate document topic distribution through document-topic similarity, and specifically comprises the following steps:
Step 3.1, constructing a learnable theme embedding matrix WhereinIs the number of topics;
Step 3.2, obtaining the topic distribution by the document level embedded representation and topic embedded matrix dot product calculation, and each enhanced document Corresponding topic distributionThe calculation mode of (a) is as follows:
;
Wherein, To enhance documentsA corresponding document-level representation of the document,In order to refer to calculating the distribution of the subject matter,The operations are normalized for the examples.
5. The method for extracting low-resource topic key subject matter facing to an online forum according to claim 4, wherein in the step 4, a semantic perception contrast learning frame is designed, dynamic negative samples are screened in the same batch of enhanced documents of the contrast learning frame, contrast learning loss is calculated, a subject embedding matrix is optimized, and subject consistency is ensured by using prior alignment loss, so that subject words are obtained, and the method specifically comprises the following steps:
step 4.1, calculating the first using the topic distribution of the enhanced document as a basis Document level embedded representation embedding corresponding to individual enhanced documentsAnd (d)Document level embedded representation embedding corresponding to individual enhanced documentsRelative semantic relevance score between:
;
Wherein, Representing the calculation of cosine similarity of two vectors;
step 4.2, screening and comparing dynamic negative samples in the reinforced documents of the same batch of the learning framework through the following formula:
;
Wherein, Super parameters that are thresholds;
Step 4.3 for Low resource documents All have low resource documentsCorresponding first enhanced documentAnd second enhancing the documentWhen training the contrast learning framework, the corpus is collected from low resourcesMedium random sampling,For batch size, take random samplesIs to enhance documents for a first batch of (a)And second batch enhanced documentsSeparately computing a first batch of enhanced document topic distributionsAnd second batch enhanced document topic distributionAnd construct positive sample pairsNegative sample pair pass within the same lotScreening, and calculating the contrast learning loss of the positive sample pair and the negative sample pair of the first batchContrast learning loss for second lot positive and negative sample pairs:
;
;
Wherein, Is a super-parameter which is used for the processing of the data,Representing temperature parameters of a document-level embedded representation, total contrast learning loss for the same batchThe method comprises the following steps:
;
Step 4.3, randomly sampling two batch sizes, namely Prior topic distributionAnd calculate a priori alignment loss:
;
Wherein, Representing a topic distribution deduced from two batches of enhanced documents,In order to be a moment order,For the number of topics currently being calculated,Is the firstThe distribution of the individual topics,Is the firstA priori topic distribution is obtained,Representing inferred topic distribution domainsThe mean value of the d-th topic of (c),Representing a priori distribution domainsThe mean value of the d-th topic,Representation ofIs the first of (2)The number of subjects to be treated,Representation ofIs the first of (2)A subject matter;
Step 4.4, total loss function :
;
Wherein, Is a super parameter.
6. The method for extracting low-resource topic key subject matter for an online forum of claim 5, wherein the step 4 of designing a semantic aware contrast learning framework training process comprises the following steps:
step T1, initializing a theme embedding matrix Configuration of dimension size including hidden variable with pre-trained language model parametersInputting the low-resource document into a large language model to generate an enhanced document set by the non-zero super parameters;
step T2, carrying out batch random extraction on the enhanced document set, inputting a contrast learning frame, and carrying out prior sampling in a Dirichlet parameter space to obtain document theme distribution of the enhanced document set;
Step T3, comprehensively evaluating the total contrast learning loss of the same batch Loss of alignment with a prioriAnd circularly executing the theme embedding matrix parameter updating operation, and stopping training when the semantic aware contrast learning framework converges.
7. The method for extracting key topics of low-resource topics for online forum as set forth in claim 6, wherein the large language model expanding and generalizing topic finding method of step 5 is specifically to give names of low-resource documents and their corresponding names in promptsThe 10 subject words corresponding to each subject are required to be generalized for each subject to generate human-readable subject insights.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202510615488.7A CN120146046A (en) | 2025-05-14 | 2025-05-14 | A low-resource topic key topic extraction method for online forums |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202510615488.7A CN120146046A (en) | 2025-05-14 | 2025-05-14 | A low-resource topic key topic extraction method for online forums |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN120146046A true CN120146046A (en) | 2025-06-13 |
Family
ID=95952626
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202510615488.7A Pending CN120146046A (en) | 2025-05-14 | 2025-05-14 | A low-resource topic key topic extraction method for online forums |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN120146046A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20220309254A1 (en) * | 2021-03-25 | 2022-09-29 | NEC Laboratories Europe GmbH | Open information extraction from low resource languages |
| US20230169271A1 (en) * | 2021-11-30 | 2023-06-01 | Adobe Inc. | System and methods for neural topic modeling using topic attention networks |
| CN117271761A (en) * | 2023-09-01 | 2023-12-22 | 东北电力大学 | Cross-language abstract summarization method based on robust self-learning strategy in low resource scenarios |
| CN118332091A (en) * | 2024-06-06 | 2024-07-12 | 中电信数智科技有限公司 | Ancient book knowledge base intelligent question-answering method, device and equipment based on large model technology |
| CN118897984A (en) * | 2024-07-18 | 2024-11-05 | 浙江财经大学 | Online human activity recognition method based on robust reinforcement learning |
| CN119360278A (en) * | 2024-10-23 | 2025-01-24 | 上海歆仁信息科技有限公司 | A dangerous behavior identification and early warning method based on multimodal analysis |
-
2025
- 2025-05-14 CN CN202510615488.7A patent/CN120146046A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20220309254A1 (en) * | 2021-03-25 | 2022-09-29 | NEC Laboratories Europe GmbH | Open information extraction from low resource languages |
| US20230169271A1 (en) * | 2021-11-30 | 2023-06-01 | Adobe Inc. | System and methods for neural topic modeling using topic attention networks |
| CN117271761A (en) * | 2023-09-01 | 2023-12-22 | 东北电力大学 | Cross-language abstract summarization method based on robust self-learning strategy in low resource scenarios |
| CN118332091A (en) * | 2024-06-06 | 2024-07-12 | 中电信数智科技有限公司 | Ancient book knowledge base intelligent question-answering method, device and equipment based on large model technology |
| CN118897984A (en) * | 2024-07-18 | 2024-11-05 | 浙江财经大学 | Online human activity recognition method based on robust reinforcement learning |
| CN119360278A (en) * | 2024-10-23 | 2025-01-24 | 上海歆仁信息科技有限公司 | A dangerous behavior identification and early warning method based on multimodal analysis |
Non-Patent Citations (1)
| Title |
|---|
| 何涛;王桂芳;杨美妮;郭楷模;: "基于词嵌入语义的精准检索式构建方法", 现代情报, no. 11, 12 November 2018 (2018-11-12) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111738004B (en) | Named entity recognition model training method and named entity recognition method | |
| CN107229610B (en) | A kind of emotional data analysis method and device | |
| CN113435211B (en) | A Text Implicit Sentiment Analysis Method Combining External Knowledge | |
| CN109101490B (en) | A factual implicit emotion recognition method and system based on fusion feature representation | |
| CN110209818A (en) | A kind of analysis method of Semantic-Oriented sensitivity words and phrases | |
| CN111126067B (en) | Entity relationship extraction method and device | |
| Liu et al. | AMFF: A new attention-based multi-feature fusion method for intention recognition | |
| CN111274790A (en) | Text-level event embedding method and device based on syntactic dependency graph | |
| CN112256866A (en) | A fine-grained sentiment analysis method for text based on deep learning | |
| CN114117070A (en) | A knowledge graph construction method, system and storage medium | |
| CN116543406A (en) | Dual-objective self-supervised text clustering method and system for medical problems based on multi-feature fusion | |
| CN117708336A (en) | A multi-strategy sentiment analysis method based on topic enhancement and knowledge distillation | |
| CN117932066A (en) | Pre-training-based 'extraction-generation' answer generation model and method | |
| CN111061873A (en) | A Multi-Channel Text Classification Method Based on Attention Mechanism | |
| CN117852523A (en) | A cross-domain small sample relationship extraction method and device for learning discriminative semantics and multi-view context | |
| Wang et al. | Incorporating linguistic knowledge for learning distributed word representations | |
| Zhou et al. | Gaussian mixture variational autoencoder for semi-supervised topic modeling | |
| CN112199503A (en) | A Chinese Text Classification Method Based on Feature Enhanced Unbalanced Bi-LSTM | |
| Guo et al. | Efficient agricultural question classification with a BERT-enhanced DPCNN model | |
| CN111339289B (en) | A Topic Model Inference Method Based on Product Reviews | |
| CN110909167B (en) | A microblog text classification system | |
| CN118446202A (en) | A method for source detection of generated text based on writing style | |
| CN118395989A (en) | Personality detection method based on hypergraph attention mechanism | |
| Gao et al. | Chinese short text classification method based on word embedding and Long Short-Term Memory Neural Network | |
| Jiang et al. | Association between community psychological label and user portrait model based on multimodal neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination |