前往小程序,Get更优阅读体验!
立即前往
首页
学习
活动
专区
工具
TVP
发布
社区首页 >专栏 >在应用大模型的场景中,我们该如何使用语义搜索?

在应用大模型的场景中,我们该如何使用语义搜索?

原创
作者头像
点火三周
修改2023-07-14 14:21:44
3.2K0
修改2023-07-14 14:21:44
举报

随着时间的推移,越来越多的企业和用户开始关注大语言模型在业务中的应用。然而,由于大语言模型中存在的过时、不准确、幻觉、一本正经的胡说八道、基于互联网数据训练这些缺点,因此,直接使用大语言模型生成的内容在商业场景中,特别是涉及到一些专业领域以及私有数据的场景,是无法提供准确或有价值的信息的。因此,大模型的应用通常需要与搜索技术相结合。由此,语义搜索也从之前的鲜有人问津,突然变成了一个热门话题。但是,将语义搜索与大型模型结合并不是一个简单的工具组合,也不是“将所有数据用机器学习模型转化 -> 再进行语义搜索 -> 最后搜索结果输入大型模型”就能实现的。因此,本文将介绍在这方面的一些思考,希望能对大家有所帮助。

我们为什么要在使用大模型时使用语义搜索?

首席,我们需要思考,为什么我们使用大模型时要使用语义搜索,知道初心是什么,然后才能坚守初心,而不至于被次要的东西所影响,最后导致项目变形。

我们的初心是用大模型的理解能力、推理能力、生成能力来帮助我们提升生产率,用户体验,但是大模型中过时的、或者不准确的、基于互联网数据而训练出来推理答案有缺陷,因此,我们需要一种技术来给大模型提供准确的上下文信息(加上提问)作为生成的原始素材。

因为语义搜索能够更好的理解长文本的对话式提问,并能从数据库中找到更相关的内容,所以,这是对大模型的一个很好补充。

语义搜索的核心能力能够为大模型提供准确的上下文(context window)

但另外一点,我们需要明确知道的是,无论是ChatGPT还是Bard,亦或是文心一言,通义千问等,LLM的输入其实都是文本。我们提交给大模型进行总结,扩展,翻译,重写,创作的内容,都不是向量

因此,准确且高效地给大语言模型提供上下文信息是主要目的而语义搜索只是一个可以选择的技术手段,而且是多路召回中的一个分支,倒排检索、数据类目和实体过滤、召回融合,重排等都是为了此目的需要考虑的技术方案。

语义搜索=向量搜索吗?

语义搜索分为稀疏表征的倒排检索和稠密表征的相似性搜索两种。我们通常说的向量搜索是指的基于embedding的稠密表征的相似性搜索(KNN和ANN搜索)但实际上,我们还有有基于稀疏表征的倒排语义检索。这两种搜索技术,都得益于大规模语言模型(注意,LLM是指参数很多的语言模型,并非特指类似chatGPT这种超高参数的模型,实际上,特征参数在数千万级别的模型都能称作大语言模型)的发展,而在召回率和相关性方面有了长足的进步,特别是在transformer的基础上演进出来的模型,比如BERT等。

但我们在选择语义搜索方案时,基于embedding的稠密表征的相似性搜索(KNN和ANN搜索)不见得就是最优解。比如说下图的SPLADE的,就是一种不逊色于大多数稠密embedding模型的稀疏表征的模型,而这种模型的优势在于,经过模型处理后的数据不是数组类型的向量,而是人类可读的、搜索可解释的词+权重,应用的仍然是倒排检索技术。因此,在模型大小,生成的索引,与检索性能上会有优势,而且因为term匹配能力强,也能更好的适应用户输入过短,没有上足够上下文时,向量搜索所表现出来的信息表达能力弱的缺点。

SPLADE
SPLADE

而在这方面,Elasticsearch中的ELSER表现优异:

向量搜索受限于什么?

当然,向量搜索仍然是具备更强大的语义理解能力的。我们需要向量搜索。但同时,也需要知道,高效、优雅、准确的构建向量搜索能力并不容易。其受限于:

  • 向量搜索在自然语言中的理解能力来自于深度学习模型,而非向量索引和向量相似性计算
    • 需要大量的计算资源和存储空间来训练和部署深度学习模型。
    • 需要大量的标注数据来训练深度学习模型。如果数据质量不高或不足以覆盖所有可能的场景,模型可能无法泛化到新的数据上。
    • 需要定期更新深度学习模型以适应数据和用户行为的变化。如果模型过时或不准确,可能会影响搜索结果的质量和用户满意度。
  • 它需要考虑向量的维度和稠密程度,以选择合适的索引和查询方法。如果向量维度过高或过低,或者向量分布不均匀,可能会影响搜索效率和准确度。
  • 向量搜索的实施和维护成本较高,涉及大量的计算资源和专业知识。对于一些资源有限的应用场景,或者缺乏专业人员对模型的选择时,这可能不是一个可行的选择。
  • 在短文本搜索的场景中,向量搜索可能会面临语义理解的挑战。虽然向量搜索可以对查询进行语义分析,但当涉及到短文本时,语义的表示和理解可能不够准确,导致结果的相关性不佳。
  • 向量搜索以词嵌入的方式表示数据,在搜索的透明性和可解释性上对人类有天然的障碍,人类即无法轻易理解两个嵌入到底第为何相似,也难以知道应该具体如何修改特征,以提升相关性;
  • embedding模型的修改、调优、再训练对于大多数的开发团队来说,门槛太高,ROI也充满了不确定性。

一句话解释,就是向量搜索的效果,并不是你选择使用向量库之后就能直接呈现,直接能够获得的。选择模型,合理调优,模型的维度、量化,模型的更新等都会影响效果。

特别是大部分向量库仅仅提供了向量存储,向量索引,向量相似性比较这三方面的能力,但这只解决了工程上问题,也就是说,向量库只解决了向量搜索搜得快不快,QPS高不高,吞吐大不大的问题,但没法解决搜索的准不准的问题,即在效果上的问题,需要依赖embedding模型的选择、调优和持续的训练。这并不是狭义上的“向量搜索”的问题,而是一个广义上的NLP(自然语言处理)的问题。

向量搜索是必选项吗?

向量搜索的相关性严重依赖于所选择的模型。从下面的测试中我们可以看到,甚至有很多embedding模型的效果还不如BM25+CE。同时也不如稀疏表征的倒排检索。因此,千万别一上来就使用基于KNN或者ANN的向量搜索,因为效果可能远没有你想象的好。而且即便你用正确的方式打开了向量搜索,获得的提升也大多在10%左右(从8x%的召回率到9x%的召回率),对最终的用户体验有多少的帮助,是需要评估的。这种评估应该来自于搜索结果的点击量方面去评估。

正确合理的使用embedding模型有哪些约束?

要使用向量搜索,我们就必须首先解决文档和query的向量化问题。也就是说,我们需要知道如何选择和使用一个embedding模型。

embedding模型的适用领域

偏科的embedding模型
偏科的embedding模型

不同的embedding模型在不同的领域是效果不一样,其原因在于:

  1. 数据特征和语义差异:不同领域的文本具有不同的特征和语义差异。例如,新闻领域的文本可能包含大量实时信息和事件,而医学领域的文本可能涉及领域特定的术语和概念。由于不同模型在训练时所使用的数据集和语料库可能存在偏差,因此在特定领域中表现较好的模型可能对其他领域的文本处理效果不佳。
  2. 数据量和多样性:嵌入模型的性能通常受到训练数据量和多样性的影响。如果某个模型在训练时使用的数据量较少或者数据不够多样化,它可能对特定领域的文本理解能力有限。相反,如果某个模型在训练时使用的数据集较大且具有广泛的覆盖范围,它通常会在不同领域中表现更好。
  3. 领域专业性:某些领域具有特定的专业性,其中的术语、概念和上下文信息对于正确理解和嵌入文本至关重要。一些模型可能对于一般性的文本表现出色,但在处理领域专业文本时可能缺乏相关知识和背景,导致效果不佳。
  4. 模型架构和训练目标:不同的嵌入模型采用不同的架构和训练目标,这可能导致它们在处理不同领域文本时的表现有所不同。例如,一些模型专注于语言模型的训练,适用于生成连续的文本;而其他模型可能更适合于分类、命名实体识别等任务。

针对特定领域,为了获得更好的效果,应考虑以下方法:

  • 使用领域特定的预训练模型:一些领域可能存在特定领域的预训练模型,这些模型在特定领域的文本处理上表现更好。也就是说,在选择一个模型时,需要了解这个模型是主要使用哪个领域的数据来训练的:
检查模型的训练数据集
检查模型的训练数据集
  • 针对特定领域进行模型微调:使用领域相关的数据对预训练模型进行微调,使其适应特定领域的特征和语义。
  • 结合多个模型或集成方法:结合多个嵌入模型的输出,或者采用集成学习方法,以获得更全面和准确的嵌入表示。

因此,如果没有适合自己数据的模型,还不如先用BM25建立自己的基准,再去一一比较效果。

token长度限制

Transformer 本身是自回归的,BERT 的创建者指出,当使用超过 512 个tokens的文档时,性能会显着下降

而基于Transormer的模型大抵都有这个限制。以下是几种基于Transformer的常见嵌入(embedding)模型:

  1. BERT(Bidirectional Encoder Representations from Transformers):BERT 是一种预训练的基于 Transformer 的模型,通过无监督学习从大规模文本数据中学习通用的语言表示。它在多个自然语言处理任务上表现出色,并且可以用于生成文本的嵌入表示。
  2. GPT(Generative Pre-trained Transformer):GPT 是一系列基于 Transformer 的模型,例如 GPT-1、GPT-2 和 GPT-3。这些模型主要用于生成文本,通过预训练的方式学习语言模型,并且可以用于生成连续的文本嵌入。
  3. XLNet(eXtreme Language Understanding Network):XLNet 是另一个基于 Transformer 的预训练模型,它采用了一种新颖的训练方法,称为排列语言模型(Permutation Language Model),以更好地捕捉句子中的依赖关系和上下文。
  4. RoBERTa(Robustly Optimized BERT Pretraining Approach):RoBERTa 是对 BERT 模型的改进和优化,通过调整训练方法和超参数等方式来提高模型的性能和鲁棒性。它也是基于 Transformer 的模型,适用于生成文本的嵌入表示。

除了上述模型,还有其他基于 Transformer 的嵌入模型,例如 Transformer-XL、Electra、BERT-Base 等。这些模型都基于 Transformer 的架构,因此,我们不仅需要根据具体任务和需求选择合适的基于 Transformer 的嵌入模型,并进行相应的预训练或微调,以获取适用于特定任务的文本嵌入表示,并且在生成嵌入的时候,还需要考虑token的限制

而对于OpenAI而言,目前,其主要的embedding模型是text-embedding-ada-002,token长度限制为:8191。参见文档

而对于其他可以从Huggingface上下载到的模型来说,大部分在model card里面都没有指明其限制,但从某些标注了限制的模型来看,一般都是限制在512以下:

代码语言:javascript
复制
SentenceTransformer(
  (0): Transformer({'max_seq_length': 509, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)

(sentence-transformers/facebook-dpr-ctx_encoder-multiset-base)

所以,如果要想用好embedding模型,需要观察自己的输入文档的大小。长文本的输入(比如论文,新闻稿等),需要采用分块(chunking)的策略来处理文件,比如,可以按行、按句子、按段落或按指定的字符数来进行分割。总体原则,就是在不超出限制的情况下,尽量保证切割出来的内容包含完整的语义。常见的处理方法有Clipping(截断法),Pooling(池化法),划窗法,压缩法。可参考:Bert 如何解决长文本问题?或者 chatgpt-retrieval-plugin/services/chunks.py

因此,embedding模型是否能够达到期望的相关性,是有非常多的限制的,比如BEIR中的dataset,都是经过合理chunk的数据

另一方面,其实不是所有的数据都应该用于embedding,参考上面的BEIR数据集,能够回答问题的部分才值得做embedding,而一篇文章中,有些部分是适合用用于精确匹配和过滤的,以论文为例:

代码语言:javascript
复制
==== Front
Clin Cancer Res
Clin Cancer Res
Clinical Cancer Research
1078-0432
1557-3265
American Association for Cancer Research

34740925
CCR-21-2595
10.1158/1078-0432.CCR-21-2595
Version of Record
Clinical Trials: Targeted Therapy
A Novel Third-generation EGFR Tyrosine Kinase Inhibitor Abivertinib for EGFR T790M-mutant Non–Small Cell Lung Cancer: a Multicenter Phase I/II Study
Efficacy and Safety of Abivertinib in Patients with T790M+ NSCLC
https://orcid.org/0000-0002-0478-176X
Zhou Qing 1
https://orcid.org/0000-0001-7078-7767
Wu Lin 2
Hu Pei 3
An Tongtong 4
Zhou Jianying 5
Zhang Li 6
Liu Xiao-Qing 7
Luo Feng 8
Zheng Xin 3
Cheng Ying 9
Yang Nong 10
Li Junling 11
Feng Jifeng 12
https://orcid.org/0000-0002-6251-3771
Han Baohui 13
Song Yong 14
Wang Kai 15
Zhang Li 16
Fang Jian 4
Zhao Hong 17
Shu Yongqian 18
Lin Xiao-Yan 19
Chen Zhihong 1
Gan Bin 1
Xu Wan-Hong 20
Tang Wei 20
Zhang Xiaoying 20
Yang Jin-Ji 1
Xu Xiao 21
https://orcid.org/0000-0002-3611-0258
Wu Yi-Long 1*
1 Department of Pulmonary Oncology, Guangdong Lung Cancer Institute, Guangdong Provincial People's Hospital and Guangdong Academy of Medical Sciences, Guangzhou, China.
2 Department of Medical Oncology, Second Chest Cancer Unit, Hunan Cancer Hospital/The Affiliated Cancer Hospital of Xiangya School of Medicine, Central South University, Changsha, China.
3 Clinical Pharmacology Research Center, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China.
4 Department of Thoracic Oncology, Peking University Cancer Hospital & Institute, Beijing, China.
5 Department of Respiratory Medicine, The First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China.
6 Department of Respiratory and Critical Care Medicine, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China.
7 Department of Lung Cancer, Fifth Medical Center of PLA General Hospital, Beijing, China.
8 Lung Cancer Center, West China Hospital of Sichuan University, Chengdu, China.
9 Division of Thoracic Oncology, Jilin Provincial Cancer Hospital, Changchun, China.
10 Department of Medical Oncology, Lung Cancer and Gastrointestinal Unit, Hunan Cancer Hospital/The Affiliated Cancer Hospital of Xiangya School of Medicine, Central South University, Changsha, China.
11 Department of Medical Oncology, China National Cancer Center/China Cancer Research Foundation/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China.
12 The Affiliated Cancer Hospital of Nanjing Medical University, Jiangsu Cancer Hospital, Jiangsu Institute of Cancer Research, Nanjing, Jiangsu, China.
13 Department of Pulmonary Medicine, Shanghai Chest Hospital, Shanghai Jiao Tong University, Shanghai 200030, China.
14 Department of Respiratory and Critical Care Medicine, Jinling Hospital, Nanjing University School of Medicine, Nanjing, China.
15 Department of Respiratory Medicine, The Second Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China.
16 Department of Medical Oncology, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Sun Yat-sen University Cancer Center, Guangzhou, China.
17 Oncology Department, Chinese PLA General Hospital, Beijing, China.
18 Department of Oncology, The First Affiliated Hospital with Nanjing Medical University, Nanjing, China.
19 Department of Medical Oncology, Fujian Medical University Union Hospital, Fuzhou, China.
20 ACEA Pharmaceutical Research, Hangzhou, China.
21 ACEA Therapeutics Inc., San Diego, California.
Trial Registration ID: NCT02330367

Prior presentation: Abstracts were accepted in World Conference on Lung Cancer, December 3–8, 2016 and American Society of Clinical Oncology, May 31- June 4, 2019; a poster was presented in American Society of Clinical Oncology, Chicago, 2019.

* Corresponding Author: Yi-Long Wu, Department of Pulmonary Oncology, Guangdong Lung Cancer Institute, Guangdong Provincial People's Hospital and Guangdong Academy of Medical Sciences, No.106 Zhongshan 2nd Road, Guangzhou 510080, China. Phone: 208-387-7855; E-mail: syylwu@live.cn
15 3 2022
14 3 2022
28 6 11271135
16 7 2021
13 9 2021
01 11 2021
©2021 The Authors; Published by the American Association for Cancer Research
2021
American Association for Cancer Research
https://creativecommons.org/licenses/by-nc-nd/4.0/ This open access article is distributed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) license.

Abstract

Purpose:

To establish recommended phase II dose (RP2D) in phase I and evaluate safety and efficacy of abivertinib in patients with EGFR Thr790Met point mutation (T790M)-positive(+) non–small cell lung cancer (NSCLC) with disease progression from prior EGFR inhibitors in phase II.

Patients and Methods:

This multicenter, open-label study included 367 adult Chinese patients. Abivertinib at doses of 50 mg twice a day to 350 mg twice a day was evaluated in phase I in continual 28-day cycles, and the RP2D of 300 mg twice a day was used in phase II in continual 21-day cycles. Primary endpoints include RP2D in phase I and objective response rate (ORR) at RP2D in phase II.

Results:

The RP2D of 300 mg twice a day for abivertinib was established based on pharmacokinetics, efficacy, and safety profiles across doses in phase I. In phase II, 227 patients received RP2D for a median treatment duration of 24.6 weeks (0.43–129). Among 209 response–evaluable patients, confirmed ORR was 52.2% [109/209; 95% confidence interval (CI): 45.2–59.1]. Disease control rate (DCR) was 88.0% (184/209; 95% CI: 82.9–92.1). The median duration of response (DoR) and progression-free survival (PFS) was 8.5 months (95% CI: 6.1–9.2) and 7.5 months (95% CI: 6.0–8.8), respectively. The median overall survival (OS) was 24.9 months [95% CI: 22.4–not reachable (NR)]. All (227/227) patients reported at least 1 adverse event (AE), with 96.9% (220/227) of treatment-related AEs. Treatment-related serious AEs were reported in 13.7% (31/227) of patients. Death was reported in 4.4% (10/227) of patients, and none was deemed as treatment-related.

Conclusions:

Abivertinib of 300 mg twice a day demonstrated favorable clinical efficacy with manageable side effects in patients with EGFR T790M+ NSCLC.

Key Lab System Project of Guangdong Science and Technology Department – Guangdong Provincial Key Lab of Translational Medicine in Lung Cancer 2017B030314120 High-level Hospital Construction Project DFJH201810 National Science and Technology Infrastructure Program National Key Science Projects Program 2018ZX09301–014–002
==== Body
pmcTranslational Relevance

This phase I/II study, investigation of the novel third-generation EGFR tyrosine kinase inhibitor (TKI) abivertinib in 367 Chinese patients with advanced EGFR Thr790Met point mutation (T790M)-positive(+) non–small cell lung cancer (NSCLC), is the first clinical study conducted in the Chinese patients at a large scale to investigate the clinical benefits of abivertinib. Current phase I/II design facilitated immediate recommended phase II dose (RP2D) assessments in an expanded patient population. Abivertinib demonstrated favorable efficacy and tolerable safety profiles in patients.

Introduction

... 

Discussion

This phase I/II study, investigation of the novel third-generation EGFR-TKI abivertinib in 367 Chinese patients with advanced NSCLC, is the first clinical study conducted in the Chinese patients at a large scale to investigate the clinical benefits of abivertinib. In phase I, 300 mg twice a day was selected as the RP2D based on favorable efficacy outcomes and a safety profile comparable with that observed at lower doses with greater steady-state PK parameters. The safety and efficacy of the 300 mg twice a day dose and schedule was further demonstrated in phase II with 227 patients. Abivertinib at 300 mg twice a day was generally well tolerated, with the most common AEs being liver transferase elevation, diarrhea, and rash, which are common with approved EGFR-TKIs and other investigational third-generation agents (9, 15–17). At the dose of 300 mg twice a day, abivertinib treatment resulted in ORR (52.2%) and DCR (88.0%) with significant DoR (8.5 months), PFS (7.5 months), and OS (24.9 months) in patients with EGFR T790M mutation. A substantial efficiency gain in abivertinib efficacy and safety evaluations is obtained by using this two-phase study design, of which a modified three-plus-three dose escalation scheme allowed quick cohort expansion and RP2D determination which was not dependent on MTD of the drug, and also this phase I/II design seamlessly facilitated immediate RP2D assessments in an expanded patient population with less toxicity and favorable efficacy.

Abivertinib demonstrated efficacious effects in overcoming T790M-mediated resistance with the ORR of 52.2%, which is comparable with other third-generation EGFR-TKIs as reported ORRs in the range between 42% and 67% from different studies (10, 15–18). Of the patients included in this study, about 30 had brain metastases. The ORRs for patients with brain metastases or without brain metastasis were 50.0% and 52.5%, respectively and among 159 patients with disease progression, 24.5% of patients were progressed in the CNS as their primary site. Given that the sample size of the patients with brain metastases in this study was small, and the cerebrospinal fluid concentration of the drug has not be systematically analyzed, the efficacy of abivertinib in patients with brain metastases is inconclusive and further study will be warranted. In phase II, abivertinib treatment demonstrated promising effects in PFS and OS with a median PFS of 7.5 months and a median OS of 24.9 months (40% maturity). The subsequent treatment after disease progression includes chemotherapy, rechallenging EGFR-TKIs and Chinese traditional medicines, etc.

In this study with Chinese patients, grade ≥ 3 AE AST/ALT increase occurred more frequently while grade ≥ 3 QT interval prolongation was observed at a similar frequency compared with data previously reported for other TKIs (2–4, 9, 19). However, the majority or AST/ALT increase AEs were mild (grade 1 in severity) and 2 AEs led to treatment discontinuation. The risk factors analysis reflected a relative actual situation for Chinese patients who had a more complicated medication history. The skin rash rate (37% for all grades and 2.2% for grade 3/4) was lower than many first- and second-generation TKIs reported (>70% for all grades and >15% for grade 3/4) probably due to spare target against WT EGFR by abivertinib treatment (2–4). ILD, a warning AE for all TKIs, occurred in 4.8% (13/272) at RP2D in this study; 11 cases (4.0%) were SAEs; 10 cases (3.7%) were grade ≥ 3; one case (0.4%) was fatal. Two cases with grade 2/3 ILD among above 13 cases were re-treated with abivertinib at reduced doses when ILD was recovered to grade 1 till disease progression, suggesting ILD is still controllable by abivertinib treatment. Together, abivertinib treatment in this study demonstrated favorable efficacy and safety profiles in patients with NSCLC patients with EGFR—resistant T790M mutation.

Abivertinib is a pyrrolpyrimidine-based, third-generation EGFR-TKI which is structurally distinct from FDA only approved third-generation TKI osimertinib (12). The molecular structure of a drug may determine its target-binding properties, subsequently the clinical outcomes, and ultimately molecular mechanism upon drug resistance. In recent clinical and nonclinical studies, resistance mechanism of abivertinib was reported to be distinct from that reviewed with osimertinb. Off-target resistance mechanisms involving TP53, MET, ERBB2, and RB1, etc. are thought to be the common resistance mechanisms found by abivertinib treatment as reported from current phase I/II study (43.3%, 13/30 cases; ref. 20), first-in-human study (62.5%, 10/16 cases; ref. 13), and nonclinical studies (BCL-2 and c-MET amplification from in vitro and in vivo results, respectively; ref. 21). EGFR tertiary mutations (8.7%, 4/30 from phase I/II and 0/16 from first-in-human resistance patients) including C797S by abivertinib treatment are much less than osimertinib resistance reported (21%–45%; refs. 13, 20). Moreover, EGFR T790M loss with abivertinib resistance (9.3%, 4/27 from phase I/II and 0/16 from first-in-human resistance patients) was less frequent than reported in osimertinib resistance cohorts (42%–68%), indicating that potential combination therapy, such as abivertinib in combination with drugs targeting bypass pathways, might be explored to overcome the resistance to abivertinib (13, 20). An impressive response was reported in several patients receiving sequential treatment firstly with abivertinib and followed by osimertinib, which suggests that rechallenging third-generation EGFR-TKIs with distinct resistance profile might warrant clinical benefits in patients failed with a third-generation EGFR-TKI (22).

In summary, phase I established abivertinib 300 mg twice a day as RP2D, which is consistent with that previously reported in abivertinib first-in-human study. The clinical efficacy of abivertinib coupled with its safety profile in phase II further confirmed positive clinical outcomes in patients with EGFR T790M+ resistant mutation. Whether the clinical benefits to abivertinib can be expanded in patients with EGFR-TKI–naïve advanced NSCLC will be evaluated in a randomized trial in comparison with gefitinib. In addition, further research into mechanism of acquired resistance with abivertinib and other third-generation EGFR-TKIs and strategies of subsequent treatment will help define the use of these agents in this setting.

Supplementary Material

Supplementary Data Click here for additional data file.

Acknowledgments

This research was supported by Key Lab System Project of Guangdong Science and Technology Department – Guangdong Provincial Key Lab of Translational Medicine in Lung Cancer (Grant No. 2017B030314120, to Y.L. Wu); High-level Hospital Construction Project (Grant No. DFJH201810, to Q. Zhou); and National Science and Technology Infrastructure Program (National Key Science Projects Program, Grant No. 2018ZX09301–014–002, to W.H. Xu, W. Tang, X. Zhang, and X. Xiao).

This study was funded by ACEA Pharmaceutical Research, Hangzhou, China. ACEA Pharmaceutical Research led the design and conduct of the study, monitored it, and provided the study drug. ACEA Pharmaceutical Research, with authors, also participated in the collection, management, analysis, and interpretation of the data. We appreciate the patients, their families, and caregivers for their participation in the study. Medical writing support was provided by Yiqun Zhou, PhD, EME Pharma Consultants, funded by ACEA Therapeutics Inc.

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.

Authors' Disclosures

Q. Zhou reports personal fees from AstraZeneca, Boehringer Ingelheim, BMS, Eli Lilly and Company, MSD, Pfizer, Sanofi, and Roche outside the submitted work. Y.L. Wu reports grants, personal fees, and other support from AstraZeneca; grants and personal fees from BMS; personal fees and other support from Boehringer Ingelheim and Merck; personal fees from Eli Lilly and Company, Hengrui, MSD, Pfizer, Roche, and Sanofi; and other support from Takeda outside the submitted work. No disclosures were reported by the other authors.

Authors' Contributions

Q. Zhou: Conceptualization, resources, investigation, writing–original draft, writing–review and editing. L. Wu: Resources, investigation, writing–review and editing. P. Hu: Resources, investigation, writing–review and editing. T. An: Resources, investigation, writing–review and editing. J. Zhou: Resources, investigation, writing–review and editing. L. Zhang: Resources, investigation, writing–review and editing. X.Q. Liu: Resources, investigation, writing–review and editing. F. Luo: Resources, investigation, writing–review and editing. X. Zheng: Resources, investigation, writing–review and editing. Y. Cheng: Resources, investigation, writing–review and editing. N. Yang: Resources, investigation, writing–review and editing. J. Li: Resources, investigation, writing–review and editing. J. Feng: Resources, investigation, writing–review and editing. B. Han: Resources, investigation, writing–review and editing. Y. Song: Resources, investigation, writing–review and editing. K. Wang: Resources, investigation, writing–review and editing. L. Zhang: Resources, investigation, writing–review and editing. J. Fang: Resources, investigation, writing–review and editing. H. Zhao: Resources, investigation, writing–review and editing. Y. Shu: Resources, investigation, writing–review and editing. X.Y. Lin: Resources, investigation, writing–review and editing. Z. Chen: Resources, investigation, writing–review and editing. B. Gan: Resources, investigation, writing–review and editing. W.H. Xu: Resources, investigation, writing–review and editing. W. Tang: Resources, investigation, writing–review and editing. X. Zhang: Resources, investigation, writing–review and editing. J.J. Yang: Resources, investigation, writing–review and editing. X. Xu: Resources, investigation, writing–review and editing. Y.L. Wu: Conceptualization, resources, investigation, writing–original draft, writing–review and editing.

Note: Supplementary data for this article are available at Clinical Cancer Research Online (http://clincancerres.aacrjournals.org/).
==== Refs
References

1. Mok TS , WuYL, ThongprasertS, YangCH, ChuDT, SaijoN, . Gefitinib or carboplatin-paclitaxel in pulmonary adenocarcinoma. N Engl J Med 2009;361 :947–57.19692680
2. Rosell R , CarcerenyE, GervaisR, VergnenegreA, MassutiB, FelipE, . Erlotinib versus standard chemotherapy as first-line treatment for European patients with advanced EGFR mutation-positive non-small-cell lung cancer (EURTAC): a multicentre, open-label, randomised phase 3 trial. Lancet Oncol 2012;13 :239–46.22285168
3. Schuler M , WuYL, HirshV, O'ByrneK, YamamotoN, MokT, . First-line afatinib versus chemotherapy in patients with non-small cell lung cancer and common epidermal growth factor receptor gene mutations and brain metastases. J Thorac Oncol 2015;11 :380–90.
4. Wu YL , ChengY, ZhouX, LeeKH, NakagawaK, NihoS, . Dacomitinib versus gefitinib as first-line treatment for patients with EGFR-mutation-positive non-small-cell lung cancer (ARCHER 1050): a randomised, open-label, phase 3 trial. Lancet Oncol 2017;18 :1454–66.28958502
5. Yu HA , ArcilaME, RekhtmanN, SimaCS, ZakowskiMF, PaoW, . Analysis of tumor specimens at the time of acquired resistance to EGFR-TKI therapy in 155 patients with EGFR-mutant lung cancers. Clin Cancer Res 2013;19 :2240–7.23470965
6. Han B , YangL, WangX, YaoLD. Efficacy of pemetrexed-based regimens in advanced non-small cell lung cancer patients with activating epidermal growth factor receptor mutations after tyrosine kinase inhibitor failure: a systematic review. Onco Targets Ther 2018;11 :2121–9.29695919
7. Reck M , Paz-AresL, BidoliP, CappuzzoF, DakhilS, Moro-SibilotD, . Outcomes in patients with aggressive or refractory disease from REVEL: A randomized phase III study of docetaxel with ramucirumab or placebo for second-line treatment of stage IV non-small-cell lung cancer. Lung Cancer 2017;112 :181–7.29191593
8. Ramalingam SS , VansteenkisteJ, PlanchardD, ChoBC, GrayJE, OheY, . Overall survival with osimertinib in untreated, EGFR-mutated advanced NSCLC. NEJM 2020;382 :41–50.31751012
9. Mok TS , WuYL, AhnMJ, GarassinoMC, KimHR, RamalingamSS, . Osimertinib or platinum-pemetrexed in EGFR T790M-positive lung cancer. N Engl J Med 2017;376 :629–40.27959700
10. Janne PA , YangJC, KimDW, PlanchardD, OheY, RamalingamSS, . AZD9291 in EGFR inhibitor-resistant non-small-cell lung cancer. N Engl J Med 2015;372 :1689–99.25923549
11. Lee CK , NovelloS, RydenA, MannH, MokT. Patient-Reported symptoms and impact of treatment with osimertinib versus chemotherapy in advanced non-small-cell lung cancer: the AURA3 trial. J Clin Oncol 2018;36 :1853–60.29733770
12. Xu X , MaoL, XuW, TangW, ZhangX, XiB, . AC0010, an irreversible EGFR inhibitor selectively targeting mutated EGFR and overcoming T790M-induced resistance in animal models and lung cancer patients. Mol Cancer Ther 2016;15 :2586–97.27573423
13. Ma Y , ZhengX, ZhaoH, FangW, ZhangY, GeJ, . First-in-human Phase I study of AC0010, a mutant-selective EGFR inhibitor in non-small cell lung cancer: Safety, efficacy, and potential mechanism of resistance. J Thorac Oncol 2018;13 :968–77.29626621
14. Biankin AV , PiantadosiS, HollingsworthSJ. Patient-centric trials for therapeutic development in precision oncology. Nature 2015;526 :361–70.26469047
15. Yang JCH , CamidgeDR, YangCT, ZhouJ, GuoR, ChiuCH, . Safety, Efficacy, and pharmacokinetics of almonertinib (HS-10296) in pretreated patients with EGFR-mutated advanced NSCLC: a multicenter, open-label, phase 1 trial. J Thorac Oncol 2020;15 :1907–18.32916310
16. Kim DW , TanDS, AixSP, SequistLV, SmitEF, HidaT, . Preliminary Phase II results of a multicenter, open-label study of nazartinib (EGF816) in adult patients with treatment-naive non-small cell lung cancer (NSCLC). J Clin Oncol 2018;36 :9094.
17. Murakami H , NokiharaH, HayashiH, SetoT, ParkK, AzumaK, . Clinical activity of ASP8273 in Asian patients with non-small cell lung cancer with EGFR activating and T790M mutations. Cancer Sci 2018;109 :2852–62.29972716
18. Sequist LV , SoriaJC, GoldmanJW, WakeleeHA, GadgeelSM, VargaA, . Rociletinib in EGFR-mutated non-small-cell lung cancer. N Engl J Med 2015;372 :1700–9.25923550
19. Yang JC , AhnMJ, KimDW, RamalingamSS, SequistLV, SuWC, . Osimertinib in pretreated T790M-positive advanced non-small-cell lung cancer: AURA study phase II extension component. J Clin Oncol 2017;35 :1288–96.28221867
20. Zhang YC , ChenZH, ZhangXC, XuCR, YanHH, XieZ, . Analysis of resistance mechanisms to abivertinib, a third-generation EGFR tyrosine kinase inhibitor, in patients with EGFR T790M-positive non-small cell lung cancer from a phase I trial. EBioMedicine 2019;43 :180–7.31027916
21. Xu WH , TangW, LiTT, ZhangXY, SunY, . Overcoming resistance to AC0010, a third generation of EGFR inhibitor, by targeting c-Met and Bcl-2. Neoplasia 2019;21 :41–45.30504063
22. Zhang YC , ZhouQ, ChenZH, ChuaiSK, YeJY, WuYL, . The spatiotemporal evoluation of EGFR C797S mutation in EGFR non-small cell lung cancer: opportunity for third-generation EGR inhibitors re-challenge. Sci Bulletin 2019;64 :499–503.

应该首先使用合适的NLP模型,提取出论文中的类目,作者,引用等信息,这种类型的信息更适合存储于keyword字段中进行全文检索和精确匹配而对于简介,总结,结果,内容,效果等内容,则进行chunk后,用embedding模型提取特征信息

embedding模型的资源消耗与利用

另一方面,为了有更好的效果,embedding模型都会倾向于增加模型中的参数以提高上下文的理解能力。一些效果好的embedding模型,甚至能够支持多种语言的语义理解,但也因为参数更多,往往消耗的资源也更大。OpenAI的embedding模型text-embedding-ada-002,其模型大小超过300GB。

如果自己使用机器学习平台进行部署,则需要注意资源消耗的问题,在Elasticsearch中,模型是在线程之间共享的。也就是说,模型会被加载到内存当中,并在多个allocation之间共享(allocation越多,并发吞吐越高),而每个allocation又能够通过配置多线程来提高处理能力(线程越多,推理速度越快)

以始为终,不忘初心:提升召回率和相关性才是最终目标

为了能给大模型提供更准确的上下文信息,需要结合不同场景下的最佳方式,才能做到高效且准确。

在实际应用中,我们往往需要结合向量搜索和其他搜索技术,甚至是结合机器学习与NLP推理技术来构建一个高效且灵活的搜索系统。这样可以充分利用各种技术的优势,同时避免各种技术的局限性。以下是一些常见的方法和建议:

  • 使用混合索引(Hybrid Index)来存储文档。混合索引是一种同时包含词项索引(Term Index)和向量索引(Vector Index)的索引结构。词项索引用于存储文档中出现的词项及其频率等信息。向量索引用于存储文档经过深度学习模型转换后得到的向量。这样可以在查询时根据不同的需求选择使用词项索引还是向量索引。
  • 使用其他NLP技术,提取出数据中的深度理解的特征与标签信息(比如类目预测和实体识别模型),以进行词索引的过滤和检索
  • 使用双层检索(Two-stage Retrieval)来执行查询。双层检索是一种先使用词项索引进行粗排(Coarse Ranking),再使用向量索引进行精排(Fine Ranking)的查询方法。粗排用于从海量文档中快速筛选出一批候选文档。精排用于从候选文档中进一步筛选出最相关的文档。这样可以在保证查询效率的同时提高查询质量。
  • 使用多路检索(Multi-stage Retrieval)来执行查询。多路检索是一种使用多个不同类型或来源的索引进行检索,并将各个检索结果进行融合(Fusion)重排(Re-ranking)的查询
  • 需要在调优相关性的同时,提供对召回效果的验证能力:
    • 调整了搜索的参数、字段、索引
    • 切换了embedding模型
    • 使用了新的排序算法

而在Elasticsearch的搜索平台中,不仅提供了提升搜索相关性的各种工具,也提供了验证召回和相关性效果的基线测试工具更得益于社区的支持,可以使用不同的插件在不同的情况下实现优化。

搜索能力的提升不可能一蹴而就,需求的变化和技术的迭代也意味着需要持续的改进。选择一个健壮、完善、被广泛验证过的平台,将是我们有效使用语义搜索,有效的与大模型相集合的良好开端,帮助我们赢在起跑线。

原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。

如有侵权,请联系 cloudcommunity@tencent.com 删除。

原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。

如有侵权,请联系 cloudcommunity@tencent.com 删除。

评论
登录后参与评论
0 条评论
热度
最新
推荐阅读
目录
  • 我们为什么要在使用大模型时使用语义搜索?
  • 语义搜索=向量搜索吗?
  • 向量搜索受限于什么?
  • 向量搜索是必选项吗?
  • 正确合理的使用embedding模型有哪些约束?
  • embedding模型的适用领域
    • token长度限制
      • embedding模型的资源消耗与利用
      • 以始为终,不忘初心:提升召回率和相关性才是最终目标
      相关产品与服务
      Elasticsearch Service
      腾讯云 Elasticsearch Service(ES)是云端全托管海量数据检索分析服务,拥有高性能自研内核,集成X-Pack。ES 支持通过自治索引、存算分离、集群巡检等特性轻松管理集群,也支持免运维、自动弹性、按需使用的 Serverless 模式。使用 ES 您可以高效构建信息检索、日志分析、运维监控等服务,它独特的向量检索还可助您构建基于语义、图像的AI深度应用。
      领券
      问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档