Backstage 自定义搜索 Collator 开发指南从脚手架生成、数据索引到搜索结果展示定制【免费下载链接】backstageBackstage is an open framework for building developer portals项目地址: https://gitcode.com/GitHub_Trending/ba/backstage搜索Search是 Backstage 开发者门户中被高频使用的核心能力而Collator收集器正是决定什么东西可以被搜索到的关键组件。本文以官方文档 Writing Custom Collators 为主线结合当前仓库中真实的模板源码与后端实现完整演示如何用一条命令脚手架出 collator 模块、实现任意数据源的索引逻辑、用TestPipeline编写测试、配置索引调度并通过前端模块定制搜索结果在页面上的呈现方式。读完本文你将具备为 Backstage 接入博客文章、文档站、内部 Wiki 或任意自定义数据源的一站式落地能力。前置认知Collator 在 Backstage Search 中的位置在动手之前先明确几个核心概念详见 Search Concepts文档Document一个可被搜索到的对象至少包含title、text、location三个字段也可以携带任意扩展字段。索引Index某一类型文档的集合。Collator定义什么可以被搜索的组件本质上是产出文档的可读对象流。一个 collator 负责一种类型文档的收集例如 Catalog Collator 索引全部实体、TechDocs Collator 索引全部技术文档见 Collators。调度SchedulerBackstage Search 采用按计划整体重建索引的策略每个 collator 可以配置独立的刷新周期。当内置的 Catalog / TechDocs collator 无法覆盖你的数据源时就可以编写自定义 collator——这也是官方文档给出的标准做法先用内置模板脚手架出模块骨架再实现自己的数据抓取逻辑。用一条命令脚手架出 Collator 模块在 Backstage 仓库根目录执行yarn new --select search-collator-module命令会提示你输入一个module ID它同时决定了包名和生成的 collator 类名。例如输入blog-posts模板会在plugins/search-backend-module-blog-posts/下创建新包自动把模块注册进后端packages/backend/src/index.ts中追加backend.add(import(internal/plugin-search-backend-module-blog-posts))。生成的包结构如下plugins/search-backend-module-blog-posts/ ├── config.d.ts ├── package.json └── src/ ├── collator/ │ ├── BlogPostsCollatorFactory.test.ts │ └── BlogPostsCollatorFactory.ts ├── index.ts └── module.ts这个模板在当前仓库中是真实存在的其源文件位于 packages/cli-module-new/templates/search-collator-module/并在 defaultTemplates.ts 中注册。模板采用 Handlebars 变量如{{collatorClass}}、{{moduleId}}来填充你输入的 module IDcollatorClass由{{ upperFirst ( camelCase moduleId ) }}CollatorFactory推导输入blog-posts即得到BlogPostsCollatorFactorymoduleVar推导出searchModuleBlogPosts之类的模块导出变量名见 portable-template.yaml。理解生成代码模块与工厂各司其职模板生成两个核心文件后端模块负责把 collator 接入搜索系统和collator 工厂负责抓取并产出文档。后端模块注册 Collator 到搜索索引src/module.ts创建一个后端模块从配置中读取可选调度缺省时回退到每 10 分钟一次的默认调度import { coreServices, createBackendModule, readSchedulerServiceTaskScheduleDefinitionFromConfig, } from backstage/backend-plugin-api; import { searchIndexRegistryExtensionPoint } from backstage/plugin-search-backend-node/alpha; import { BlogPostsCollatorFactory } from ./collator/BlogPostsCollatorFactory; const DEFAULT_SCHEDULE { frequency: { minutes: 10 }, timeout: { minutes: 15 }, initialDelay: { seconds: 3 }, }; export const searchModuleBlogPosts createBackendModule({ pluginId: search, moduleId: blog-posts-collator, register({ registerInit }) { registerInit({ deps: { config: coreServices.rootConfig, logger: coreServices.logger, scheduler: coreServices.scheduler, indexRegistry: searchIndexRegistryExtensionPoint, }, async init({ config, logger, scheduler, indexRegistry }) { const scheduleConfig config .getOptionalConfig(search.collators.blogPosts) ?.getOptionalConfig(schedule); const schedule scheduleConfig ? readSchedulerServiceTaskScheduleDefinitionFromConfig(scheduleConfig) : DEFAULT_SCHEDULE; indexRegistry.addCollator({ schedule: scheduler.createScheduledTaskRunner(schedule), factory: BlogPostsCollatorFactory.fromConfig(config, { logger }), }); }, }); }, });几个值得留意的实现细节与 module.ts.hbs 完全一致pluginId: search表示这是挂在 search 插件上的扩展模块searchIndexRegistryExtensionPoint是 search-backend-node 暴露的扩展点其接口SearchIndexRegistryExtensionPoint只包含addCollator与addDecorator两个方法见 alpha.ts配置键search.collators.blogPosts的路径由{{camelCase moduleId}}推导即模块 ID 的驼峰形式DEFAULT_SCHEDULE三要素frequency每 10 分钟、timeout15 分钟、initialDelay3 秒。在 Search 插件侧扩展点注册的 collator 会被SearchIndexRegistry收集起来addCollator只是 push 进数组随后在插件初始化时通过searchIndexService.init({ searchEngine, collators, decorators })交给索引服务统一调度与重建见 plugins/search-backend/src/plugin.ts 与 L127-L137。Collator 工厂实现 DocumentCollatorFactory 接口src/collator/BlogPostsCollatorFactory.ts实现DocumentCollatorFactory接口。核心是execute()这个 async generator——它逐条 yield 出IndexableDocument对象每个文档必须包含title、text、location三个字段import { LoggerService } from backstage/backend-plugin-api; import { Config } from backstage/config; import { DocumentCollatorFactory, IndexableDocument, } from backstage/plugin-search-common; import { Readable } from node:stream; export type BlogPostsCollatorFactoryOptions { logger: LoggerService; }; export class BlogPostsCollatorFactory implements DocumentCollatorFactory { public readonly type blog-posts; private readonly logger: LoggerService; static fromConfig( _config: Config, options: BlogPostsCollatorFactoryOptions, ): BlogPostsCollatorFactory { return new BlogPostsCollatorFactory(options); } private constructor(options: BlogPostsCollatorFactoryOptions) { this.logger options.logger; } async getCollator(): PromiseReadable { return Readable.from(this.execute()); } private async *execute(): AsyncGeneratorIndexableDocument { this.logger.info(Collating documents for blog-posts); // TODO: Replace with your data fetching logic yield* []; } }两个关键点type属性blog-posts是文档类型标识前端需要用它与搜索结果进行类型匹配后面定制结果展示一节会用到getCollator()返回Readable通过Readable.from(this.execute())把 async generator 包装成 Node 可读流。execute()内的yield* []是占位符等待替换为真实的数据抓取逻辑。当前仓库中内置 Collator 的典型实现也遵循同一模式例如 Catalog Collatorplugins/search-backend-module-catalog/src/collators与 TechDocs Collatorplugins/search-backend-module-techdocs/src/collators可作为参考范本。实现 Collator填充你的数据抓取逻辑要让 collator 真正可用需要把execute()里的占位逻辑替换为真实的数据抓取。下面示例从一个内部 API 拉取博客文章import { LoggerService } from backstage/backend-plugin-api; import { Config } from backstage/config; import { DocumentCollatorFactory, IndexableDocument, } from backstage/plugin-search-common; import { Readable } from node:stream; type BlogPost { id: string; title: string; body: string; author: string; }; export type BlogPostsCollatorFactoryOptions { logger: LoggerService; }; export class BlogPostsCollatorFactory implements DocumentCollatorFactory { public readonly type blog-posts; private readonly baseUrl: string; private readonly logger: LoggerService; static fromConfig( config: Config, options: BlogPostsCollatorFactoryOptions, ): BlogPostsCollatorFactory { const baseUrl config.getString(blogPosts.baseUrl); return new BlogPostsCollatorFactory(baseUrl, options); } private constructor( baseUrl: string, options: BlogPostsCollatorFactoryOptions, ) { this.baseUrl baseUrl; this.logger options.logger; } async getCollator(): PromiseReadable { return Readable.from(this.execute()); } private async *execute(): AsyncGeneratorIndexableDocument { this.logger.info(Collating documents for blog-posts); const response await fetch(${this.baseUrl}/blog-posts); const posts: BlogPost[] await response.json(); for (const post of posts) { yield { title: post.title, text: post.body, location: /blog-posts/${post.id}, }; } } }要点说明fromConfig从配置读取blogPosts.baseUrlconfig.getString是必填读取若缺失会直接抛错这与模块中可选调度配置getOptionalConfig形成对比你可以按数据源的实际需求自由选择读取方式location是搜索结果的跳转地址通常是门户内相对路径用户点击搜索结果时会被解析为可访问链接text字段是全文检索的主要来源建议放入可读正文内容title是结果标题。大数据集用游标分页避免内存爆炸:::tip对于大体积数据集请在execute()中使用基于游标的分页避免一次性把所有记录加载进内存private async *execute(): AsyncGeneratorIndexableDocument { let cursor: string | undefined undefined; do { const url cursor ? ${this.baseUrl}/blog-posts?cursor${cursor} : ${this.baseUrl}/blog-posts; const response await fetch(url); const { items, nextCursor } await response.json(); for (const item of items) { yield { title: item.title, text: item.body, location: /blog-posts/${item.id}, }; } cursor nextCursor; } while (cursor); }:::这个模式do...whilenextCursor同样是 模板注释中给出的官方建议每页抓取后把nextCursor回填到查询参数直到服务端不再返回下一页为止。async generator 的惰性求值保证文档按需产出配合下游索引器边读边写内存占用与数据集大小解耦。测试 Collator用 TestPipeline 验证产出模板同时生成了测试文件src/collator/BlogPostsCollatorFactory.test.ts它使用backstage/plugin-search-backend-node提供的TestPipeline来运行 collator 并校验输出。按你的实现更新测试即可import { BlogPostsCollatorFactory } from ./BlogPostsCollatorFactory; import { mockServices } from backstage/backend-test-utils; import { TestPipeline } from backstage/plugin-search-backend-node; const mockPosts [ { id: 1, title: Getting Started, body: Welcome to our engineering blog, author: Alice, }, { id: 2, title: Best Practices, body: Tips for writing great code, author: Bob, }, ]; describe(BlogPostsCollatorFactory, () { beforeEach(() { global.fetch jest.fn().mockResolvedValue({ json: async () mockPosts, }); }); it(returns a collator with the correct type, async () { const factory BlogPostsCollatorFactory.fromConfig( mockServices.rootConfig({ data: { blogPosts: { baseUrl: http://localhost } }, }), { logger: mockServices.logger.mock() }, ); expect(factory.type).toBe(blog-posts); }); it(runs the collator and returns documents, async () { const factory BlogPostsCollatorFactory.fromConfig( mockServices.rootConfig({ data: { blogPosts: { baseUrl: http://localhost } }, }), { logger: mockServices.logger.mock() }, ); const collator await factory.getCollator(); const { error, documents } await TestPipeline.fromCollator( collator, ).execute(); expect(error).toBeUndefined(); expect(documents).toHaveLength(2); expect(documents[0]).toMatchObject({ title: Getting Started, text: Welcome to our engineering blog, location: /blog-posts/1, }); }); });测试技巧与原理mockServices.rootConfig/mockServices.logger.mock()来自backstage/backend-test-utils用于在测试环境注入最小化的配置与日志服务mockglobal.fetch通过jest.fn().mockResolvedValue(...)拦截网络请求使测试不依赖真实外部服务TestPipeline.fromCollator(collator).execute()返回{ error, documents }——error表示管线中是否出现异常documents是被收集的全部产出文档见 TestPipeline 源码。TestPipeline还支持fromDecorator/fromIndexer分别测试装饰器与索引器是搜索侧单元测试的统一工具箱。配置索引调度每 10 分钟还是 6 小时生成的模块会从app-config.yaml读取可选调度配置未配置时 collator 默认每 10 分钟运行一次。自定义调度示例search: collators: blogPosts: schedule: # same options as in SchedulerServiceTaskScheduleDefinition # supports cron, ISO duration, human duration as used in code initialDelay: { seconds: 90 } # supports cron, ISO duration, human duration as used in code frequency: { hours: 6 } # supports ISO duration, human duration as used in code timeout: { minutes: 3 }解析流程是模块先getOptionalConfig(search.collators.blogPosts)再取其中的schedule子配置交给readSchedulerServiceTaskScheduleDefinitionFromConfig解析如果配置不存在则使用DEFAULT_SCHEDULE。这里值得注意三点三个字段的取值风格initialDelay、frequency支持 cron、ISO 时长与人类可读时长如{ hours: 6 }timeout仅支持 ISO 时长与人类可读时长timeout应大于单次索引预计耗时否则调度器会按超时处理frequency决定索引刷新频率需要根据数据源更新频率权衡——数据源更新快用短周期反之拉长周期可减少后端负载同一search.collators命名空间下不同 collator 可各自配置独立的调度实现差异化刷新这也与 Collators 文档 中内置 collator 的配置方式一致。定制搜索结果展示为你的文档类型做一个专属结果项自定义 collator 产出的搜索结果会自动以默认列表项形式展示。若要定制呈现效果需要创建一个前端模块来扩展 search 插件。脚手架前端模块在 Backstage 根目录执行yarn new --select frontend-plugin-module按提示输入插件 ID 填search模块 ID 填blog-posts。模板会在plugins/search-module-blog-posts/下创建新包并把它作为依赖写入packages/app/package.json新前端系统会自动发现并加载该模块。创建结果列表项组件编写一个渲染单条搜索结果的组件。每条结果携带来自 collator 的title、text、location字段import { Link } from backstage/core-components; import ListItemIcon from material-ui/core/ListItemIcon; import ListItemText from material-ui/core/ListItemText; import { SearchDocument } from backstage/plugin-search-common; import { ReactNode } from react; export interface BlogPostSearchResultListItemProps { icon?: ReactNode; result?: SearchDocument; rank?: number; } export function BlogPostSearchResultListItem( props: BlogPostSearchResultListItemProps, ) { const { icon, result } props; if (!result) return null; return ( {icon ListItemIcon{icon}/ListItemIcon} ListItemText primaryTypographyProps{{ variant: h6 }} primary{ Link noTrack to{result.location} {result.title} /Link } secondary{result.text} / / ); }组件通过result.location作为跳转目标result.title作为链接文本result.text作为摘要并支持可选的icon与rank参数。用 SearchResultListItemBlueprint 注册结果项更新生成的src/module.tsx通过SearchResultListItemBlueprint注册结果列表项扩展并提供一个predicate谓词来匹配来自你 collator 的结果。谓词检查结果的type字段它必须与 collator 工厂中设置的type属性一致——本例即blog-postsimport { createFrontendModule } from backstage/frontend-plugin-api; import { SearchResultListItemBlueprint } from backstage/plugin-search-react/alpha; const blogPostSearchResultListItem SearchResultListItemBlueprint.make({ name: blog-posts, params: { predicate: result result.type blog-posts, component: () import(./components/BlogPostSearchResultListItem).then( m m.BlogPostSearchResultListItem, ), }, }); export const searchModuleBlogPosts createFrontendModule({ pluginId: search, extensions: [blogPostSearchResultListItem], });理解这个 blueprints 的机制源码见 SearchResultListItemBlueprint.tsxpredicate是可选参数默认谓词恒为 true渲染所有类型的结果一旦提供则只有返回 true 的结果才由该扩展渲染。因此多个自定义 collator 可以各注册一个带专属predicate的结果项互不冲突component接收一个返回 Promise 的加载函数blueprint 内部用lazy()做代码分割保证结果项组件按需加载即上例中的import(...).then(...)blueprint 的attachTo指向page:search的items输入这正是搜索结果列表的扩展挂载点组件最终会被包进ExtensionBoundary与内置的SearchResultListItemExtension自动获得rank、result、noTrack等属性的传递noTrack可通过扩展配置关闭点击追踪。关键路径速查用途仓库相对路径官方指南本文主体docs/features/search/custom-collators.md内置 Collator 总览docs/features/search/collators.md搜索核心概念docs/features/search/concepts.md脚手架模板源文件packages/cli-module-new/templates/search-collator-module/模板注册清单packages/cli-module-new/src/lib/defaultTemplates.ts索引注册扩展点接口plugins/search-backend-node/src/alpha.tsSearch 插件收集与初始化 collatorplugins/search-backend/src/plugin.tsTestPipeline 测试工具plugins/search-backend-node/src/test-utils/TestPipeline.ts前端结果项 Blueprintplugins/search-react/src/alpha/blueprints/SearchResultListItemBlueprint.tsx至此一条完整的自定义搜索能力链路已经打通后端用脚手架生成 collator 模块并实现数据抓取与调度用TestPipeline保障产出正确性前端用 frontend-plugin-module 模板注册专属结果列表项通过type谓词把自定义文档类型渲染成想要的形态。后续若需要进一步定制还可以在此基础上研究 Decorator文档装饰器为索引附加元数据、或接入 Elasticsearch 等外部搜索引擎见 search-engines.md让搜索体验更贴合你的组织需求。【免费下载链接】backstageBackstage is an open framework for building developer portals项目地址: https://gitcode.com/GitHub_Trending/ba/backstage创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考