写在前面
本文将在第五篇《整合ElasticSearch实现商品搜索》的基础上,使用ElasticSearch实现商品复杂搜索这一功能。
中文分词器
由于商品搜索涉及到中文搜索,因此ElasticSearch需要安装分词器才可以支持。前面我们安装的分词器是IKAnalyzer,接下来简单学习如何使用它。
默认分词器
使用默认分词器,只是将中文逐字进行分割,并不符合我们的要求:
1 2 3 4 5
| GET /pms/_analyze { "text": "华为手机使用较为丝滑", "tokenizer": "standard" }
|
输出结果:
中文分词器
使用中文分词器后,可以将中文文本按照语境进行分隔,可以满足我们的要求:
1 2 3 4 5
| GET /pms/_analyze { "text": "华为手机使用较为丝滑", "tokenizer": "ik_max_word" }
|
输出结果:
其实在前一文中,我们就在EsProduct中,对于需要进行中文分词的字段,都使用了@Field
注解且将analyzer属性设置为ik_max_word
:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
| /** * 商品名称 */ @Field(analyzer = "ik_max_word",type = FieldType.Text) private String name;
/** * 副标题 */ @Field(analyzer = "ik_max_word",type = FieldType.Text) private String subTitle;
/** * 关键字 */ @Field(analyzer = "ik_max_word",type = FieldType.Text) private String keywords;
|
商品简单搜索
商品简单搜索即通过商品名称、副标题或者关键字来搜索包含指定关键字的商品。
使用Query DSL来调用ES的Restful API来实现
1 2 3 4 5 6 7 8 9 10 11 12 13
| GET /pms/product/_search { "from": 0, "size": 5, "query": { "multi_match": { "query": "小米", "fields": [ "name","subTitle","keywords" ] } } }
|
执行结果如下:
在SpringBoot中使用Elasticsearch Repository的衍生查询来实现
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
| public interface EsProductRepository extends ElasticsearchRepository<EsProduct,Long> { /** * 搜索查询 * @param name 商品名称 * @param subTitle 商品副标题 * @param keywords 商品关键字 * @param page 分页信息 * @return 搜索结果 */ Page<EsProduct> findByNameOrSubTitleOrKeywords(String name, String subTitle, String keywords, Pageable page); }
@Service public class EsProductServiceImpl implements EsProductService { @Autowired private EsProductRepository esProductRepository;
@Override public Page<EsProduct> search(String keyword, Integer pageNum, Integer pageSize) { Pageable pageable = PageRequest.of(pageNum,pageSize); return esProductRepository.findByNameOrSubTitleOrKeywords(keyword,keyword,keyword,pageable); } }
|
Elasticsearch Repository的衍生查询
Elasticsearch Repository的衍生查询原理其实很简单,就是将一定规则名称的方法转化为Elasticsearch的Query DSL语句,可以查看如下图:
复杂商品搜索
复杂商品搜索会涉及到过滤、不同字段匹配权重以及排序等功能。
这里的需求是按照输入的关键字来搜索商品名称、副标题和关键字,可以按照品牌和分类进行筛选,可以有5种排序方式,默认按相关度进行排序。查看一下接口文档:
这里有一些比较特殊的需求,如商品名称匹配关键字的商品,我们认为与搜索条件更匹配,其次是副标题和关键字,此时就需要使用到function_score
查询。
在ElasticSearch中,搜索到文档的相关性由_score
字段来表示,文档的_score
字段值越高,表示与搜索条件越匹配。而function_score
查询可通过设置权重来影响_score
字段值,因此使用function_score
查询就可以很好的满足我们的需求。
这里我们设置商品名称权重为10,商品副标题权重为5,商品关键字权重为2。
使用Query DSL来调用ES的Restful API来实现
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
| POST /pms/product/_search { "query": { "function_score": { "query": { "bool": { "must": [ {"match_all": {}} ], "filter": { "bool": { "must" :[ { "term": { "brandId": 6 } }, { "term": { "productCategoryId": 19 } } ] } } } }, "functions": [ { "filter": { "match": { "name": "小米" } }, "weight": 10 }, { "filter": { "match": { "subTitle": "小米" } }, "weight": 5 }, { "filter": { "match": { "keywords": "小米" } }, "weight": 2 } ], "score_mode": "sum", "min_score": 2 } }, "sort": [ { "_score": { "order": "desc" } } ] }
|
执行结果如下:
在SpringBoot中使用Elasticsearch Repository的衍生查询来实现
回到EsProductService接口,我们在里面新定义一个方法:
1 2 3 4 5 6 7 8 9 10 11
| /** * 根据关键字搜索名称、副标题或者关键字复合查询 * @param keyword 关键字 * @param brandId 品牌id * @param productCategoryId 商品类别id * @param pageNum 页码 * @param pageSize 每页数量 * @param sort 排序字段 * @return 搜索结果 */ Page<EsProduct> search(String keyword,Long brandId,Long productCategoryId,Integer pageNum, Integer pageSize,Integer sort);
|
然后在EsProductServiceImpl
类中实现这个search方法,注意此时需要开发者自定义查询条件QueryBuilder
:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61
| @Override public Page<EsProduct> search(String keyword, Long brandId, Long productCategoryId, Integer pageNum, Integer pageSize, Integer sort) { Pageable pageable = PageRequest.of(pageNum,pageSize); NativeSearchQueryBuilder nativeSearchQueryBuilder = new NativeSearchQueryBuilder(); //分页 nativeSearchQueryBuilder.withPageable(pageable); //过滤 if(brandId != null || productCategoryId != null){ BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery(); if(brandId != null){ boolQueryBuilder.must(QueryBuilders.termQuery("brandId",brandId)); } if(productCategoryId != null){ boolQueryBuilder.must(QueryBuilders.termQuery("productCategoryId",productCategoryId)); } nativeSearchQueryBuilder.withFilter(boolQueryBuilder); } //搜索 if(StringUtils.isEmpty(keyword)){ nativeSearchQueryBuilder.withQuery(QueryBuilders.matchAllQuery()); }else{ List<FunctionScoreQueryBuilder.FilterFunctionBuilder> filterFunctionBuilders = new ArrayList<>(); filterFunctionBuilders.add(new FunctionScoreQueryBuilder.FilterFunctionBuilder(QueryBuilders.matchQuery("name",keyword), ScoreFunctionBuilders.weightFactorFunction(10)));
filterFunctionBuilders.add(new FunctionScoreQueryBuilder.FilterFunctionBuilder(QueryBuilders.matchQuery("subTitle",keyword), ScoreFunctionBuilders.weightFactorFunction(5)));
filterFunctionBuilders.add(new FunctionScoreQueryBuilder.FilterFunctionBuilder(QueryBuilders.matchQuery("keywords",keyword), ScoreFunctionBuilders.weightFactorFunction(2)));
FunctionScoreQueryBuilder.FilterFunctionBuilder[] builders = new FunctionScoreQueryBuilder.FilterFunctionBuilder[filterFunctionBuilders.size()]; filterFunctionBuilders.toArray(builders); FunctionScoreQueryBuilder functionScoreQueryBuilder = QueryBuilders.functionScoreQuery(builders) .scoreMode(FunctionScoreQuery.ScoreMode.SUM) .setMinScore(2); nativeSearchQueryBuilder.withQuery(functionScoreQueryBuilder); } //排序 if(sort == 1){ //按新品从新到旧 nativeSearchQueryBuilder.withSort(SortBuilders.fieldSort("id").order(SortOrder.DESC)); }else if(sort == 2){ //按销量从高到低 nativeSearchQueryBuilder.withSort(SortBuilders.fieldSort("sale").order(SortOrder.DESC)); }else if(sort == 3){ //按价格从低到高 nativeSearchQueryBuilder.withSort(SortBuilders.fieldSort("price").order(SortOrder.ASC)); }else if(sort == 4){ //按价格从高到低 nativeSearchQueryBuilder.withSort(SortBuilders.fieldSort("price").order(SortOrder.DESC)); }else{ //按相关度 nativeSearchQueryBuilder.withSort(SortBuilders.scoreSort().order(SortOrder.DESC)); } //不传则默认按照相关度由高到低排 nativeSearchQueryBuilder.withSort(SortBuilders.scoreSort().order(SortOrder.DESC)); NativeSearchQuery searchQuery = nativeSearchQueryBuilder.build(); LOGGER.info("DSL:{}", searchQuery.getQuery().toString()); return esProductRepository.search(searchQuery); }
|
然后在EsProductController
类中定义一个名为search的方法:
1 2 3 4 5 6 7 8 9 10 11 12 13
| @ApiOperation(value = "综合搜索、筛选、排序") @ApiImplicitParam(name = "sort", value = "排序字段:0->按相关度;1->按新品;2->按销量;3->价格从低到高;4->价格从高到低", defaultValue = "0", allowableValues = "0,1,2,3,4", paramType = "query", dataType = "integer") @GetMapping("/search") public CommonResult<CommonPage<EsProduct>> search(@RequestParam(required = false) @ApiParam("关键字") String keyword, @RequestParam(required = false) @ApiParam("品牌id") Long brandId, @RequestParam(required = false) @ApiParam("商品类别id") Long productCategoryId, @RequestParam(required = false, defaultValue = "0")@ApiParam("页码") Integer pageNum, @RequestParam(required = false, defaultValue = "5")@ApiParam("每页数量") Integer pageSize, @RequestParam(required = false, defaultValue = "0") @ApiParam("排序字段")Integer sort) { Page<EsProduct> esProductPage = esProductService.search(keyword,brandId,productCategoryId,pageNum, pageSize,sort); return CommonResult.success(CommonPage.restPage(esProductPage)); }
|
相关商品推荐(猜你喜欢)
当我们在查看某个商品的时候,底部一般会有一些商品推荐(猜你喜欢),这里我们选择使用ElasticSearch来简单实现。
这里的需求是,可以根据指定商品的id来查找相关商品,查看一下接口文档:
也就是说我们这里的原理是这样的:首先根据商品id获取到指定商品的信息,然后以指定商品的名称、品牌、分类来搜索商品,并且过滤掉当前的商品(剔除此商品的id),调整搜索条件中的权重以获取最好的匹配度。
这里我们设置商品名称权重为8,商品副标题权重为2,商品关键字权重为2,商品品牌id权重为5,商品商品类别id权重为3。
使用Query DSL来调用ES的Restful API来实现
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69
| POST /pms/product/_search { "query": { "function_score": { "query": { "bool": { "must": [ { "match_all": {} } ], "filter": { "bool": { "must_not": { "term": { "id": 28 } } } } } }, "functions": [ { "filter": { "match": { "name": "红米5A" } }, "weight": 8 }, { "filter": { "match": { "subTitle": "红米5A" } }, "weight": 2 }, { "filter": { "match": { "keywords": "红米5A" } }, "weight": 2 }, { "filter": { "term": { "brandId": 6 } }, "weight": 5 }, { "filter": { "term": { "productCategoryId": 19 } }, "weight": 3 } ], "score_mode": "sum", "min_score": 2 } } }
|
执行结果如下:
在SpringBoot中使用Elasticsearch Repository的衍生查询来实现
回到EsProductService
接口,我们在里面新定义一个方法:
1 2 3 4 5 6 7 8
| /** * 根据商品id推荐相关商品 * @param id 商品id * @param pageNum 页码 * @param pageSize 每页数量 * @return */ Page<EsProduct> recommend(Long id, Integer pageNum, Integer pageSize);
|
然后在EsProductServiceImpl
类中实现这个recommend方法,注意此时需要开发者自定义查询条件QueryBuilder
:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46
| @Override public Page<EsProduct> recommend(Long id, Integer pageNum, Integer pageSize) { Pageable pageable = PageRequest.of(pageNum,pageSize); List<EsProduct> esProductList = esProductDao.getAllEsProductList(id); if(esProductList.size()>0){ EsProduct esProduct = esProductList.get(0); String keyword = esProduct.getName(); Long brandId = esProduct.getBrandId(); Long productCategoryId = esProduct.getProductCategoryId(); //根据商品标题、品牌、分类进行搜索 List<FunctionScoreQueryBuilder.FilterFunctionBuilder> filterFunctionBuilders = new ArrayList<>(); filterFunctionBuilders.add(new FunctionScoreQueryBuilder.FilterFunctionBuilder(QueryBuilders.matchQuery("name",keyword), ScoreFunctionBuilders.weightFactorFunction(8)));
filterFunctionBuilders.add(new FunctionScoreQueryBuilder.FilterFunctionBuilder(QueryBuilders.matchQuery("subTitle",keyword), ScoreFunctionBuilders.weightFactorFunction(2)));
filterFunctionBuilders.add(new FunctionScoreQueryBuilder.FilterFunctionBuilder(QueryBuilders.matchQuery("keywords",keyword), ScoreFunctionBuilders.weightFactorFunction(2)));
filterFunctionBuilders.add(new FunctionScoreQueryBuilder.FilterFunctionBuilder(QueryBuilders.matchQuery("brandId",brandId), ScoreFunctionBuilders.weightFactorFunction(5)));
filterFunctionBuilders.add(new FunctionScoreQueryBuilder.FilterFunctionBuilder(QueryBuilders.matchQuery("productCategoryId",productCategoryId), ScoreFunctionBuilders.weightFactorFunction(3)));
FunctionScoreQueryBuilder.FilterFunctionBuilder[] builders = new FunctionScoreQueryBuilder.FilterFunctionBuilder[filterFunctionBuilders.size()]; filterFunctionBuilders.toArray(builders); FunctionScoreQueryBuilder functionScoreQueryBuilder = QueryBuilders.functionScoreQuery(builders) .scoreMode(FunctionScoreQuery.ScoreMode.SUM) .setMinScore(2); //过滤掉相同的商品 BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder(); boolQueryBuilder.mustNot(QueryBuilders.termQuery("id",id));
//构建查询条件 NativeSearchQueryBuilder builder = new NativeSearchQueryBuilder(); builder.withQuery(functionScoreQueryBuilder); builder.withFilter(boolQueryBuilder); builder.withPageable(pageable); NativeSearchQuery searchQuery = builder.build(); LOGGER.info("DSL:{}", searchQuery.getQuery().toString()); return esProductRepository.search(searchQuery); } return new PageImpl<>(null); }
|
然后在EsProductController
类中定义一个名为recommend的方法:
1 2 3 4 5 6 7 8
| @ApiOperation(value = "根据商品id推荐商品") @GetMapping( "/recommend/{id}") public CommonResult<CommonPage<EsProduct>> recommend(@PathVariable @ApiParam("商品id") Long id, @RequestParam(required = false, defaultValue = "0")@ApiParam("页码") Integer pageNum, @RequestParam(required = false, defaultValue = "5")@ApiParam("每页数量") Integer pageSize) { Page<EsProduct> esProductPage = esProductService.recommend(id, pageNum, pageSize); return CommonResult.success(CommonPage.restPage(esProductPage)); }
|
聚合搜索商品相关信息
在搜索商品时,通常会有一个筛选界面来帮助用户快速找到想要的商品,这里使用Elasticsearch来简单实现。
这里的需求是,可以根据搜索关键字获取到与关键字匹配商品相关的分类,品牌以及属性,如下图:
这里可以使用ElasticSearch的聚合来实现,搜索出相关商品,聚合出商品的品牌、分类以及属性,取出现次数最多的前10个即可。
使用Query DSL来调用ES的Restful API来实现
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64
| POST /pms/product/_search { "query": { "multi_match": { "query": "华为", "fields": [ "name", "subTitle", "keywords" ] } }, "size": 0, "aggs": { "brandNames": { "terms": { "field": "brandName", "size": 10 } }, "productCategoryNames": { "terms": { "field": "productCategoryName", "size": 10 } }, "allAttrValues": { "nested": { "path": "attrValueList" }, "aggs": { "productAttrs": { "filter": { "term": { "attrValueList.type": 1 } }, "aggs": { "attrIds": { "terms": { "field": "attrValueList.productAttributeId", "size": 10 }, "aggs": { "attrValues": { "terms": { "field": "attrValueList.value", "size": 10 } }, "attrNames": { "terms": { "field": "attrValueList.name", "size": 10 } } } } } } } } } }
|
执行结果如下,当我们搜索“华为”这个关键字时,聚合出了如下的分类和品牌信息:
聚合出了“屏幕尺寸”、“网络”、“系统”等筛选属性信息:
在SpringBoot中使用Elasticsearch Repository的衍生查询来实现
在SpringBoot中实现聚合操作非常复杂,已经超出了Elasticsearch Repository的使用范围,需要直接使用更为底层的ElasticsearchTemplate
来实现。
第一步,在com.kenbings.shop.shopelasticsearch
包内定义一个名为domain的包,并在domain包内定义一个名为EsProductRelatedInfo
的类,这是搜索相关商品品牌名称,分类名称及属性:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
| /** * 搜索相关商品品牌名称,分类名称及属性 */ @Data public class EsProductRelatedInfo { private List<String> brandNames; private List<String> productCategoryNames; private List<ProductAttr> productAttrs;
@Data public static class ProductAttr{ private Long attrId; private String attrName; private List<String> attrValues; } }
|
第二步,回到EsProductService
接口,我们在里面新定义一个方法:
1 2 3 4
| /** * 获取搜索词相关品牌、分类、属性 */ EsProductRelatedInfo searchRelatedInfo(String keyword);
|
第三步,在EsProductServiceImpl
类中实现这个searchRelatedInfo
方法,注意此时需要开发者自定义查询条件QueryBuilder
:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75
| @Autowired private ElasticsearchTemplate elasticsearchTemplate;
@Override public EsProductRelatedInfo searchRelatedInfo(String keyword) { NativeSearchQueryBuilder builder = new NativeSearchQueryBuilder(); //搜索条件 if(StringUtils.isEmpty(keyword)){ builder.withQuery(QueryBuilders.matchAllQuery()); }else{ builder.withQuery(QueryBuilders.multiMatchQuery(keyword,"name","subTitle","keywords")); } //聚合搜索品牌名称 builder.addAggregation(AggregationBuilders.terms("brandNames").field("brandName")); //集合搜索分类名称 builder.addAggregation(AggregationBuilders.terms("productCategoryNames").field("productCategoryName")); //聚合搜索商品属性,去除type=1的属性 AbstractAggregationBuilder aggregationBuilder = AggregationBuilders.nested("allAttrValues","attrValueList") .subAggregation(AggregationBuilders.filter("productAttrs",QueryBuilders.termQuery("attrValueList.type",1)) .subAggregation(AggregationBuilders.terms("attrIds") .field("attrValueList.productAttributeId") .subAggregation(AggregationBuilders.terms("attrValues") .field("attrValueList.value")) .subAggregation(AggregationBuilders.terms("attrNames") .field("attrValueList.name")))); builder.addAggregation(aggregationBuilder); NativeSearchQuery searchQuery = builder.build(); SearchResponse searchResponse = elasticsearchTemplate.query(searchQuery, response -> response); LOGGER.info("DSL:{}",searchQuery.getQuery().toString()); return convertProductRelatedInfo(searchResponse); }
/** * 将返回结果转换为对象 */ private EsProductRelatedInfo convertProductRelatedInfo(SearchResponse searchResponse) { EsProductRelatedInfo productRelatedInfo = new EsProductRelatedInfo(); Map<String, Aggregation> aggregationMap = searchResponse.getAggregations().getAsMap(); //设置品牌 Aggregation brandNames = aggregationMap.get("brandNames"); List<String> brandNameList = new ArrayList<>(); for(int i = 0; i<((Terms) brandNames).getBuckets().size(); i++){ brandNameList.add(((Terms) brandNames).getBuckets().get(i).getKeyAsString()); } productRelatedInfo.setBrandNames(brandNameList); //设置分类 Aggregation productCategoryNames = aggregationMap.get("productCategoryNames"); List<String> productCategoryNameList = new ArrayList<>(); for(int i=0;i<((Terms) productCategoryNames).getBuckets().size();i++){ productCategoryNameList.add(((Terms) productCategoryNames).getBuckets().get(i).getKeyAsString()); } productRelatedInfo.setProductCategoryNames(productCategoryNameList); //设置参数 Aggregation productAttrs = aggregationMap.get("allAttrValues"); List<? extends Terms.Bucket> attrIds = ((LongTerms) ((Filter) ((Nested) productAttrs).getAggregations().get("productAttrs")).getAggregations().get("attrIds")).getBuckets(); List<EsProductRelatedInfo.ProductAttr> attrList = new ArrayList<>(); for (Terms.Bucket attrId : attrIds) { EsProductRelatedInfo.ProductAttr attr = new EsProductRelatedInfo.ProductAttr(); attr.setAttrId((Long) attrId.getKey()); List<String> attrValueList = new ArrayList<>(); List<? extends Terms.Bucket> attrValues = ((StringTerms) attrId.getAggregations().get("attrValues")).getBuckets(); List<? extends Terms.Bucket> attrNames = ((StringTerms) attrId.getAggregations().get("attrNames")).getBuckets(); for (Terms.Bucket attrValue : attrValues) { attrValueList.add(attrValue.getKeyAsString()); } attr.setAttrValues(attrValueList); if(!CollectionUtils.isEmpty(attrNames)){ String attrName = attrNames.get(0).getKeyAsString(); attr.setAttrName(attrName); } attrList.add(attr); } productRelatedInfo.setProductAttrs(attrList); return productRelatedInfo; }
|
第四步,在EsProductController
类中定义一个名为searchRelatedInfo
的方法:
1 2 3 4 5 6
| @ApiOperation(value = "获取搜索的相关品牌、分类及筛选属性") @GetMapping(value = "/search/relate") public CommonResult<EsProductRelatedInfo> searchRelatedInfo(@RequestParam(required = false) @ApiParam("关键字") String keyword) { EsProductRelatedInfo productRelatedInfo = esProductService.searchRelatedInfo(keyword); return CommonResult.success(productRelatedInfo); }
|
接口测试
启动项目,访问Swagger-UI接口文档地址,即浏览器访问http://localhost:8080/swagger-ui.html
链接,可以看到新的接口已经出现了:
然后进行接口测试。首先后台用户进行登录,接着测试相关接口,可以看到接口都是正常的:
这样本篇关于使用ElasticSearch实现商品复杂搜索的学习就完成了,后续介绍如何整合MongoDB实现用户商品浏览记录这一功能。本篇笔记源码,可以点击 这里 进行阅读。