FiftyOne 集成 Pinecone为计算机视觉数据构建向量相似性检索实战指南【免费下载链接】fiftyoneRefine high-quality datasets and visual AI models项目地址: https://gitcode.com/GitHub_Trending/fi/fiftyone导读本文基于 docs/source/integrations/pinecone.rst系统讲解如何在 FiftyOne 中集成 Pinecone 向量搜索引擎为计算机视觉数据集构建可复用的相似性索引similarity index。读完本文你将掌握从环境配置、认证、创建/连接索引到样本级与目标框patch级相似性查询、索引动态增删向量、以及通过sort_by_similarity与 App 进行可视化检索的完整实战能力。Pinecone 是目前最流行的向量搜索引擎之一FiftyOne 已将其作为内置的相似性检索后端backend让开发者可以像使用本地 sklearn 索引一样把海量图像/视频的 embedding 托管到 Pinecone 云端索引并获得毫秒级相似性查询能力。Pinecone FiftyOne核心价值与基本工作流FiftyOne 的 Brain 模块fiftyone.brain提供了统一的相似性计算 API。当设置backendpinecone后compute_similarity()会负责创建 Pinecone 索引、批量上传向量sort_by_similarity()则负责执行查询。整个过程既可以在 Python 中以编程方式完成也可以在 FiftyOne App 中通过点选操作完成。Pinecone 后端同样支持自然语言文本检索只要底层的 embedding 模型支持 text prompt例如 CLIP 系列模型你就能用一句自然语言描述去检索视觉数据这依赖 FiftyOne 的 文本相似性检索能力sort_by_similarity(querya photo of a dog)。基本工作流Basic Recipe在 FiftyOne 数据集上使用 Pinecone 建立相似性索引并执行查询的标准流程如下将数据集加载到 FiftyOne为样本或目标框patches计算 embedding 向量或选择一个模型用于自动生成 embedding调用compute_similarity()设置backendpinecone并指定自定义brain_key生成 Pinecone 相似性索引使用sort_by_similarity()基于该索引查询数据如不再需要删除索引。下面是一个完整的演示示例代码需配合 Pinecone 账号与pinecone-client客户端使用见下一节import fiftyone as fo import fiftyone.brain as fob import fiftyone.zoo as foz # Step 1: Load your data into FiftyOne dataset foz.load_zoo_dataset(quickstart) # Steps 2 and 3: Compute embeddings and create a similarity index brain_key pinecone_index pinecone_index fob.compute_similarity( dataset, brain_keybrain_key, backendpinecone, )索引生成后通过brain_key即可查询数据# Step 4: Query your data query dataset.first().id # query by sample ID view dataset.sort_by_similarity( query, brain_keybrain_key, k10, # limit to 10 most similar samples ) # Step 5 (optional): Cleanup # Delete the Pinecone index pinecone_index dataset.load_brain_results(brain_key) pinecone_index.cleanup() # Delete run record from FiftyOne dataset.delete_brain_run(brain_key)注意compute_similarity()在未显式指定model或embeddings时会自动为quickstart这样的内置数据集计算默认 embedding在真实项目中建议显式传入模型或预计算向量详见创建相似性索引一节。环境准备与 Setup安装 Pinecone 客户端使用 Pinecone 后端前必须先安装 Pinecone 官方 Python 客户端pip install pinecone-client选择 Pinecone 后端默认情况下调用compute_similarity()或sort_by_similarity()时 FiftyOne 使用本地 sklearn 后端。要切换到 Pinecone只需把backend参数设为pineconeimport fiftyone.brain as fob fob.compute_similarity(..., backendpinecone, ...)如果希望永久默认使用 Pinecone可以通过环境变量设置export FIFTYONE_BRAIN_DEFAULT_SIMILARITY_BACKENDpinecone或者在 brain 配置文件~/.fiftyone/brain_config.json中设置default_similarity_backend参数{ default_similarity_backend: pinecone }认证Authentication连接 Pinecone 服务需要提供凭据FiftyOne 支持三种配置方式。方式一环境变量推荐将凭据存入环境变量FiftyOne 在建立连接时自动读取export FIFTYONE_BRAIN_SIMILARITY_PINECONE_API_KEYXXXXXX # Serverless indexes无服务器索引 export FIFTYONE_BRAIN_SIMILARITY_PINECONE_CLOUDaws export FIFTYONE_BRAIN_SIMILARITY_PINECONE_REGIONus-east-1 # Pod-based indexesPod 型索引 export FIFTYONE_BRAIN_SIMILARITY_PINECONE_ENVIRONMENTus-east-1-aws方式二FiftyOne Brain 配置将凭据写入~/.fiftyone/brain_config.json注意该文件默认不存在需要手动创建{ similarity_backends: { pinecone: { api_key: XXXXXXXXXXXX, cloud: aws, # serverless indexes region: us-east-1, # serverless indexes environment: us-east-1-aws # pod-based indexes } } }方式三关键字参数在每次调用compute_similarity()等方法时手动传入凭据import fiftyone.brain as fob pinecone_index fob.compute_similarity( ... backendpinecone, brain_keypinecone_index, api_keyXXXXXX, cloudaws, regionus-east-1, )使用该方式时后续通过load_brain_results()加载索引也必须手动补全凭据pinecone_index dataset.load_brain_results( pinecone_index, api_keyXXXXXX, cloudaws, regionus-east-1, )Pinecone 配置参数详解Pinecone 后端支持以下查询/建索引参数可自定义相似性索引行为参数默认值说明index_nameNone要使用或创建的 Pinecone 索引名称不指定时自动生成唯一名称index_typeNone实际默认serverless新建索引的类型支持[serverless, pod]namespaceNone存储向量所使用的命名空间namespacemetriccosine索引使用的距离/相似度度量支持(cosine, dotproduct, euclidean)replicasNone新建 Pod 型索引时的副本数可选shardsNone新建 Pod 型索引时的分片数可选podsNone新建 Pod 型索引时的 Pod 数量可选pod_typeNone新建 Pod 型索引时的 Pod 类型可选参数细节如 Pod 规格、Serverless 与 Pod 的差异可参考 Pinecone 官方索引文档本仓库文档仅约定其默认行为与可选取值。这些参数可以通过上文任何一种策略传入。例如在 brain 配置中定义一个 serverless 索引{ similarity_backends: { pinecone: { index_name: your-index, index_type: serverless, metric: cosine } } }更常见的做法是在compute_similarity()中直接传参来配置具体的新索引pinecone_index fob.compute_similarity( ... backendpinecone, brain_keypinecone_index, index_nameyour-index, index_typeserverless, metriccosine, )管理 Brain RunsFiftyOne 为管理相似性计算产生的 brain run 提供了一套完整方法它们定义在 fiftyone/core/collections.py 的SampleCollection中。列出 brain runs使用 list_brain_runs() 查看数据集上已有的 brain key并可按类型、patches 字段、是否支持文本 prompt 等条件过滤import fiftyone.brain as fob # List all brain runs dataset.list_brain_runs() # Only list similarity runs dataset.list_brain_runs(typefob.Similarity) # Only list specific similarity runs dataset.list_brain_runs( typefob.Similarity, patches_fieldground_truth, supports_promptsTrue, )获取 run 信息使用 get_brain_info() 读取某个 brain run 的配置信息info dataset.get_brain_info(brain_key) print(info)加载结果使用load_brain_results()加载某个 run 对应的相似性索引实例返回SimilarityIndex对象pinecone_index dataset.load_brain_results(brain_key)重命名 run使用 rename_brain_run() 修改已有 run 的 brain keydataset.rename_brain_run(brain_key, new_brain_key)删除 run 记录使用 delete_brain_run() 删除 run 记录dataset.delete_brain_run(brain_key)⚠️ 重要delete_brain_run()只删除FiftyOne 数据集中的 brain run记录不会删除与之关联的 Pinecone 云端索引。要真正删除 Pinecone 索引需要显式调用# Delete the Pinecone index pinecone_index dataset.load_brain_results(brain_key) pinecone_index.cleanup()实战示例常见向量检索工作流以下示例均假设已按上文配置好 Pinecone API 凭据。创建相似性索引创建新的 Pinecone 索引时需要向compute_similarity()传入embeddings或model参数有四种典型方式import fiftyone as fo import fiftyone.brain as fob import fiftyone.zoo as foz dataset foz.load_zoo_dataset(quickstart) model_name clip-vit-base32-torch model foz.load_zoo_model(model_name) brain_key pinecone_index # Option 1: Compute embeddings on the fly from model name fob.compute_similarity( dataset, modelmodel_name, backendpinecone, brain_keybrain_key, ) # Option 2: Compute embeddings on the fly from model instance fob.compute_similarity( dataset, modelmodel, backendpinecone, brain_keybrain_key, ) # Option 3: Pass precomputed embeddings as a numpy array embeddings dataset.compute_embeddings(model) fob.compute_similarity( dataset, embeddingsembeddings, backendpinecone, brain_keybrain_key, ) # Option 4: Pass precomputed embeddings by field name dataset.compute_embeddings(model, embeddings_fieldembeddings) fob.compute_similarity( dataset, embeddingsembeddings, backendpinecone, brain_keybrain_key, )创建索引时还可以把上一节介绍的配置参数作为额外 kwargs 传入以定制索引行为。创建 Patch目标框相似性索引通过指定patches_field参数可以为数据集中的对象目标框如检测框建立相似性索引import fiftyone as fo import fiftyone.brain as fob import fiftyone.zoo as foz dataset foz.load_zoo_dataset(quickstart) fob.compute_similarity( dataset, patches_fieldground_truth, modelclip-vit-base32-torch, backendpinecone, brain_keypinecone_patches, )连接已有索引如果数据集样本/目标框的向量已经存在于某个 Pinecone 索引中可以传入index_name直接连接而无需重新计算 embeddingimport fiftyone as fo import fiftyone.brain as fob import fiftyone.zoo as foz dataset foz.load_zoo_dataset(quickstart) fob.compute_similarity( dataset, modelclip-vit-base32-torch, # zoo model used (if applicable) embeddingsFalse, # dont compute embeddings index_nameyour-index, # the existing Pinecone index brain_keypinecone_index, backendpinecone, )向索引增删向量当数据集发生增删、需要让 Pinecone 索引与数据集保持一致时可使用add_to_index()与remove_from_index()import numpy as np import fiftyone as fo import fiftyone.brain as fob import fiftyone.zoo as foz dataset foz.load_zoo_dataset(quickstart) pinecone_index fob.compute_similarity( dataset, modelclip-vit-base32-torch, brain_keypinecone_index, backendpinecone, ) print(pinecone_index.total_index_size) # 200 view dataset.take(10) ids view.values(id) # Delete 10 samples from a dataset dataset.delete_samples(view) # Delete the corresponding vectors from the index pinecone_index.remove_from_index(sample_idsids) # Add 20 samples to a dataset samples [fo.Sample(filepathtmp%d.jpg % i) for i in range(20)] sample_ids dataset.add_samples(samples) # Add corresponding embeddings to the index embeddings np.random.rand(20, 512) pinecone_index.add_to_index(embeddings, sample_ids) print(pinecone_index.total_index_size) # 210从索引中检索向量使用get_embeddings()可按 ID 从 Pinecone 索引批量取回向量方便导出、迁移或做进一步分析import fiftyone as fo import fiftyone.brain as fob import fiftyone.zoo as foz dataset foz.load_zoo_dataset(quickstart) pinecone_index fob.compute_similarity( dataset, modelclip-vit-base32-torch, brain_keypinecone_index, backendpinecone, ) # Retrieve embeddings for the entire dataset ids dataset.values(id) embeddings, sample_ids, _ pinecone_index.get_embeddings(sample_idsids) print(embeddings.shape) # (200, 512) print(sample_ids.shape) # (200,) # Retrieve embeddings for a view ids dataset.take(10).values(id) embeddings, sample_ids, _ pinecone_index.get_embeddings(sample_idsids) print(embeddings.shape) # (10, 512) print(sample_ids.shape) # (10,)查询 Pinecone 索引在任意数据集或视图上追加sort_by_similarity()视图阶段即可执行相似性查询。该方法的签名与语义定义在 fiftyone/core/collections.py其query支持以下四种形式一个样本/目标框的ID与索引维度一致的查询向量一组ID 列表一段文本描述要求模型支持 text prompt。import numpy as np import fiftyone as fo import fiftyone.brain as fob import fiftyone.zoo as foz dataset foz.load_zoo_dataset(quickstart) fob.compute_similarity( dataset, modelclip-vit-base32-torch, brain_keypinecone_index, backendpinecone, ) # Query by vector query np.random.rand(512) # matches the dimension of CLIP embeddings view dataset.sort_by_similarity(query, k10, brain_keypinecone_index) # Query by sample ID query dataset.first().id view dataset.sort_by_similarity(query, k10, brain_keypinecone_index) # Query by a list of IDs query [dataset.first().id, dataset.last().id] view dataset.sort_by_similarity(query, k10, brain_keypinecone_index) # Query by text prompt query a photo of a dog view dataset.sort_by_similarity(query, k10, brain_keypinecone_index)重要行为说明在DatasetView上执行相似性搜索时只会从该视图中返回结果如果视图包含未纳入索引的样本这些样本永远不会出现在结果中。这意味着你可以只对整个数据集索引一次然后通过构造视图在任意子集上反复检索。直接访问 Pinecone 客户端SimilarityIndex实例的index属性暴露了底层 Pinecone 客户端可直接调用其原生方法import fiftyone as fo import fiftyone.brain as fob import fiftyone.zoo as foz dataset foz.load_zoo_dataset(quickstart) pinecone_index fob.compute_similarity( dataset, modelclip-vit-base32-torch, brain_keypinecone_index, backendpinecone, ) print(pinecone_index.index)高级用法定制索引 部分数据建索引将自定义参数与先建索引、后补充向量的模式结合可以精细控制索引内容。例如自定义索引名、改用点积相似度、只对数据子集填充向量import fiftyone as fo import fiftyone.brain as fob import fiftyone.zoo as foz dataset foz.load_zoo_dataset(quickstart) # Create a custom Pinecone index pinecone_index fob.compute_similarity( dataset, modelclip-vit-base32-torch, embeddingsFalse, # well add embeddings below metricdotproduct, brain_keypinecone_index, backendpinecone, index_namecustom-pinecone-index, ) # Add embeddings for a subset of the dataset view dataset.take(10) embeddings, sample_ids, _ pinecone_index.compute_embeddings(view) pinecone_index.add_to_index(embeddings, sample_ids) print(pinecone_index.index)源码层面的调用关系佐证上述 API 并非孤立存在其底层实现在仓库中均有对应sort_by_similarity()实现在 fiftyone/core/collections.py它通过fos.SortBySimilarity(query, kk, reversereverse, dist_fielddist_field, brain_keybrain_key)构造视图阶段stage最终返回一个DatasetView。该方法还支持reverse按最不相似排序与dist_field把距离写入样本字段两个可选参数。list_brain_runs()、rename_brain_run()、get_brain_info()、load_brain_results()、delete_brain_run()分别定义在 fiftyone/core/collections.py它们统一委托给fob.BrainMethod即 fiftyone/core/brain.py 中定义的 brain run 框架完成 run 的注册、查询与删除。compute_embeddings()定义在 fiftyone/core/collections.py是compute_similarity()内部计算向量的基础能力。Pinecone 后端的SimilarityIndex含add_to_index、remove_from_index、get_embeddings、cleanup、total_index_size等成员由compute_similarity()返回可通过dataset.load_brain_results(brain_key)重新加载——这正是先索引、后查询、随时增删这一完整闭环的关键。小结与最佳实践凭据优先用环境变量FIFTYONE_BRAIN_SIMILARITY_PINECONE_*系列环境变量可被 FiftyOne 自动读取避免在代码中硬编码密钥若用关键字参数传凭据后续load_brain_results()也要重复传入。区分索引与记录delete_brain_run()只清记录pinecone_index.cleanup()才真正销毁云端索引请按需调用以免误删或遗留资源。一次索引、多次检索对整个数据集建索引一次之后在任意视图上执行sort_by_similarity()即可获得子集内的高效相似性检索。文本检索很实用使用 CLIP 等支持 text prompt 的模型时一句自然语言即可驱动 Pinecone 完成视觉数据检索非常适合数据清洗与标注辅助场景。【免费下载链接】fiftyoneRefine high-quality datasets and visual AI models项目地址: https://gitcode.com/GitHub_Trending/fi/fiftyone创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考