首页
关于
Search
1
联想510s mini 安装 Ventura
966 阅读
2
基于K8s + Jenkins+Docker + Gitee 自动部署 - 配置 Jenkins Item + Gitee Webhook (二)
600 阅读
3
Spring Boot Schedule疑问及线程池配置
496 阅读
4
Server Send Events With Spring Boot
439 阅读
5
Ngrok使用自有服务器服务器及域名 - 解决Mac client问题
424 阅读
默认分类
SSH
typecho
Spring boot
其他
mysql
k8s
jenkins
docker
Java
mqtt
MongoDB
登录
/
注册
Search
标签搜索
k8s
docker
ssh
mysql
db
gitee
jenkins
ngrok
黑苹果
MQTT
CC
累计撰写
19
篇文章
累计收到
1
条评论
首页
栏目
默认分类
SSH
typecho
Spring boot
其他
mysql
k8s
jenkins
docker
Java
mqtt
MongoDB
页面
关于
搜索到
19
篇与
cc
的结果
2024-08-01
Mongo错误修复 - WiredTiger metadata corruption detected
免责声明本文提供的所有信息仅供参考,作者不对使用本文所含信息而导致的任何直接或间接损失负责。在进行MongoDB数据恢复操作之前,请务必备份您的数据并仔细阅读MongoDB官方文档。数据恢复操作具有风险,可能导致数据的永久丢失或损坏。作者建议在执行任何数据恢复步骤之前,先在测试环境中进行尝试,并咨询有经验的数据库管理员或技术专家。背景系统: macOS 13.6Mongo: 5.0.3 (Docker 容器部署)原因: 由于Docker Desktop在Mac上运行消耗资源非常高,所以又安装了OrbStack进行docker管理。 在OrbStack运行时,意外的打开了Docker Desktop(噩耗的开始)。 刚开始没在意,中午吃饭发现系统刷新失败了!? 果然还是出问题了。782c0bd0dfb4 mongo:5.0.3 "docker-entrypoint.s…" 6 weeks ago Restarting (14) 46 seconds ago mongodbmongodb的容器一直在重启,于是看看log:{"t":{"$date":"2024-08-01T06:15:22.352+00:00"},"s":"I", "c":"NETWORK", "id":4915701, "ctx":"-","msg":"Initialized wire specification","attr":{"spec":{"incomingExternalClient":{"minWireVersion":0,"maxWireVersion":13},"incomingInternalClient":{"minWireVersion":0,"maxWireVersion":13},"outgoing":{"minWireVersion":0,"maxWireVersion":13},"isInternalClient":true}}} {"t":{"$date":"2024-08-01T06:15:22.352+00:00"},"s":"I", "c":"CONTROL", "id":23285, "ctx":"-","msg":"Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'"} {"t":{"$date":"2024-08-01T06:15:22.354+00:00"},"s":"W", "c":"ASIO", "id":22601, "ctx":"main","msg":"No TransportLayer configured during NetworkInterface startup"} {"t":{"$date":"2024-08-01T06:15:22.355+00:00"},"s":"I", "c":"NETWORK", "id":4648601, "ctx":"main","msg":"Implicit TCP FastOpen unavailable. If TCP FastOpen is required, set tcpFastOpenServer, tcpFastOpenClient, and tcpFastOpenQueueSize."} {"t":{"$date":"2024-08-01T06:15:22.355+00:00"},"s":"W", "c":"ASIO", "id":22601, "ctx":"main","msg":"No TransportLayer configured during NetworkInterface startup"} {"t":{"$date":"2024-08-01T06:15:22.355+00:00"},"s":"I", "c":"REPL", "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"TenantMigrationDonorService","ns":"config.tenantMigrationDonors"}} {"t":{"$date":"2024-08-01T06:15:22.355+00:00"},"s":"I", "c":"REPL", "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"TenantMigrationRecipientService","ns":"config.tenantMigrationRecipients"}} {"t":{"$date":"2024-08-01T06:15:22.356+00:00"},"s":"I", "c":"CONTROL", "id":4615611, "ctx":"initandlisten","msg":"MongoDB starting","attr":{"pid":1,"port":27017,"dbPath":"/data/db","architecture":"64-bit","host":"2f51c27fb1c7"}} {"t":{"$date":"2024-08-01T06:15:22.356+00:00"},"s":"I", "c":"CONTROL", "id":23403, "ctx":"initandlisten","msg":"Build Info","attr":{"buildInfo":{"version":"5.0.3","gitVersion":"657fea5a61a74d7a79df7aff8e4bcf0bc742b748","openSSLVersion":"OpenSSL 1.1.1f 31 Mar 2020","modules":[],"allocator":"tcmalloc","environment":{"distmod":"ubuntu2004","distarch":"x86_64","target_arch":"x86_64"}}}} {"t":{"$date":"2024-08-01T06:15:22.356+00:00"},"s":"I", "c":"CONTROL", "id":51765, "ctx":"initandlisten","msg":"Operating System","attr":{"os":{"name":"Ubuntu","version":"20.04"}}} {"t":{"$date":"2024-08-01T06:15:22.356+00:00"},"s":"I", "c":"CONTROL", "id":21951, "ctx":"initandlisten","msg":"Options set by command line","attr":{"options":{"net":{"bindIp":"*"}}}} {"t":{"$date":"2024-08-01T06:15:22.357+00:00"},"s":"I", "c":"STORAGE", "id":22270, "ctx":"initandlisten","msg":"Storage engine to use detected by data files","attr":{"dbpath":"/data/db","storageEngine":"wiredTiger"}} {"t":{"$date":"2024-08-01T06:15:22.357+00:00"},"s":"I", "c":"STORAGE", "id":22315, "ctx":"initandlisten","msg":"Opening WiredTiger","attr":{"config":"create,cache_size=3455M,session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),builtin_extension_config=(zstd=(compression_level=6)),file_manager=(close_idle_time=600,close_scan_interval=10,close_handle_minimum=250),statistics_log=(wait=0),verbose=[recovery_progress,checkpoint_progress,compact_progress],"}} {"t":{"$date":"2024-08-01T06:15:22.776+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1722492922:776094][1:0x7f18c7605c80], file:WiredTiger.wt, connection: __wt_btree_tree_open, 637: unable to read root page from file:WiredTiger.wt: WT_ERROR: non-specific WiredTiger error"}} {"t":{"$date":"2024-08-01T06:15:22.776+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1722492922:776189][1:0x7f18c7605c80], file:WiredTiger.wt, connection: __wt_btree_tree_open, 643: WiredTiger has failed to open its metadata: WT_ERROR: non-specific WiredTiger error"}} {"t":{"$date":"2024-08-01T06:15:22.776+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1722492922:776210][1:0x7f18c7605c80], file:WiredTiger.wt, connection: __wt_btree_tree_open, 646: This may be due to the database files being encrypted, being from an older version or due to corruption on disk: WT_ERROR: non-specific WiredTiger error"}} {"t":{"$date":"2024-08-01T06:15:22.776+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1722492922:776232][1:0x7f18c7605c80], file:WiredTiger.wt, connection: __wt_btree_tree_open, 649: You should confirm that you have opened the database with the correct options including all encryption and compression options: WT_ERROR: non-specific WiredTiger error"}} {"t":{"$date":"2024-08-01T06:15:22.783+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1722492922:783621][1:0x7f18c7605c80], file:WiredTiger.wt, connection: __wt_btree_tree_open, 637: unable to read root page from file:WiredTiger.wt: WT_ERROR: non-specific WiredTiger error"}} {"t":{"$date":"2024-08-01T06:15:22.783+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1722492922:783671][1:0x7f18c7605c80], file:WiredTiger.wt, connection: __wt_btree_tree_open, 643: WiredTiger has failed to open its metadata: WT_ERROR: non-specific WiredTiger error"}} {"t":{"$date":"2024-08-01T06:15:22.783+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1722492922:783683][1:0x7f18c7605c80], file:WiredTiger.wt, connection: __wt_btree_tree_open, 646: This may be due to the database files being encrypted, being from an older version or due to corruption on disk: WT_ERROR: non-specific WiredTiger error"}} {"t":{"$date":"2024-08-01T06:15:22.783+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1722492922:783688][1:0x7f18c7605c80], file:WiredTiger.wt, connection: __wt_btree_tree_open, 649: You should confirm that you have opened the database with the correct options including all encryption and compression options: WT_ERROR: non-specific WiredTiger error"}} {"t":{"$date":"2024-08-01T06:15:22.790+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1722492922:790000][1:0x7f18c7605c80], file:WiredTiger.wt, connection: __wt_btree_tree_open, 637: unable to read root page from file:WiredTiger.wt: WT_ERROR: non-specific WiredTiger error"}} {"t":{"$date":"2024-08-01T06:15:22.790+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1722492922:790045][1:0x7f18c7605c80], file:WiredTiger.wt, connection: __wt_btree_tree_open, 643: WiredTiger has failed to open its metadata: WT_ERROR: non-specific WiredTiger error"}} {"t":{"$date":"2024-08-01T06:15:22.790+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1722492922:790057][1:0x7f18c7605c80], file:WiredTiger.wt, connection: __wt_btree_tree_open, 646: This may be due to the database files being encrypted, being from an older version or due to corruption on disk: WT_ERROR: non-specific WiredTiger error"}} {"t":{"$date":"2024-08-01T06:15:22.790+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1722492922:790063][1:0x7f18c7605c80], file:WiredTiger.wt, connection: __wt_btree_tree_open, 649: You should confirm that you have opened the database with the correct options including all encryption and compression options: WT_ERROR: non-specific WiredTiger error"}} {"t":{"$date":"2024-08-01T06:15:22.791+00:00"},"s":"W", "c":"STORAGE", "id":22347, "ctx":"initandlisten","msg":"Failed to start up WiredTiger under any compatibility version. This may be due to an unsupported upgrade or downgrade."} {"t":{"$date":"2024-08-01T06:15:22.791+00:00"},"s":"W", "c":"STORAGE", "id":22348, "ctx":"initandlisten","msg":"WiredTiger metadata corruption detected"} {"t":{"$date":"2024-08-01T06:15:22.791+00:00"},"s":"F", "c":"STORAGE", "id":50944, "ctx":"initandlisten","msg":"Please read the documentation for starting MongoDB with --repair here: http://dochub.mongodb.org/core/repair"} {"t":{"$date":"2024-08-01T06:15:22.791+00:00"},"s":"F", "c":"-", "id":23091, "ctx":"initandlisten","msg":"Fatal assertion","attr":{"msgid":50944,"file":"src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp","line":685}} {"t":{"$date":"2024-08-01T06:15:22.791+00:00"},"s":"F", "c":"-", "id":23092, "ctx":"initandlisten","msg":"\n\n***aborting after fassert() failure\n\n"} 关键信息: Failed to start up WiredTiger under any compatibility version. This may be due to an unsupported upgrade or downgrade. WiredTiger metadata corruption detected Please read the documentation for starting MongoDB with --repair here: http://dochub.mongodb.org/core/repair1、"WiredTiger metadata corruption detected" 原因清楚了,存储引擎的元数据坏掉了。在 MongoDB 中,WiredTiger 是一个用于存储引擎的名字。WiredTiger 引擎提供了高效的数据压缩和并发处理能力,从 MongoDB 3.2 版本开始成为默认的存储引擎。WiredTiger 文件实际上包含了数据文件和索引文件。下面是一些有关 WiredTiger 文件的关键点: 存储引擎:WiredTiger 是 MongoDB 的存储引擎,它负责管理数据库的数据存储、索引、并发控制和缓存等方面。 数据文件:在 MongoDB 数据库目录中,WiredTiger 存储引擎会创建一系列文件,用于存储数据和索引。这些文件包括 .wt 后缀的文件。 配置文件:WiredTiger.wt 是一个主要的配置文件,它包含了有关存储引擎配置和元数据的信息。 日志文件:WiredTiger 还使用一组日志文件(如 WiredTigerLog 开头的文件)来进行写入操作的日志记录,以便在发生崩溃时进行数据恢复。 压缩和并发:WiredTiger 提供了数据压缩功能,可以显著减少存储空间需求。此外,它还支持多线程并发写入,从而提高数据库性能。 简而言之,WiredTiger 是 MongoDB 用于管理和存储数据的核心组件,提供了高效的数据存储和管理能力。2、"http://dochub.mongodb.org/core/repair"问题发生时系统没有数据写入,所以我们数据应该没问题,根据提示我们进行修复:mongod --dbpath /data/db --repair修复由于在容器中部署的Mongo,不可能在这个容器中进行修复,所以重新起一个同版本的容器进行修复。步骤:备份数据确定Mongo 版本启动修复Mongo的容器"fix-mongo"拷贝数据到"fix-mongo"并修复从"fix-mongo"导出修复好的数据数据备份zip -r bad_mongodata_yyyyMMdd.zip /your/mount/path/mongodata确定Mongo版本 docker inspect your_container |grep 'MONGO_VERSION' "MONGO_VERSION=5.0.3"我这里是"5.0.3"启动修复Mongo的容器"fix-mongo"docker run --name fix-mongo -d mongo:your_mongo_version拷贝数据到"fix-mongo"并修复将整个数据目录拷贝到"fix-mongo"的"/tmp"## 拷贝数据到容器 docker cp /your/mount/path/mongodata fix-mongo:/tmp ## 修复 - 等待结束 mongod --dbpath /tmp/mongodata --repair ## 修复的部分log ... {"t":{"$date":"2024-08-01T06:47:54.435+00:00"},"s":"I", "c":"STORAGE", "id":22319, "ctx":"initandlisten","msg":"Finished shutting down session sweeper thread"} {"t":{"$date":"2024-08-01T06:47:54.451+00:00"},"s":"I", "c":"STORAGE", "id":4795902, "ctx":"initandlisten","msg":"Closing WiredTiger","attr":{"closeConfig":"leak_memory=true,"}} {"t":{"$date":"2024-08-01T06:47:54.693+00:00"},"s":"I", "c":"STORAGE", "id":4795901, "ctx":"initandlisten","msg":"WiredTiger closed","attr":{"durationMillis":242}} {"t":{"$date":"2024-08-01T06:47:54.693+00:00"},"s":"I", "c":"STORAGE", "id":22279, "ctx":"initandlisten","msg":"shutdown: removing fs lock..."} {"t":{"$date":"2024-08-01T06:47:54.693+00:00"},"s":"I", "c":"-", "id":4784931, "ctx":"initandlisten","msg":"Dropping the scope cache for shutdown"} {"t":{"$date":"2024-08-01T06:47:54.693+00:00"},"s":"I", "c":"FTDC", "id":20626, "ctx":"initandlisten","msg":"Shutting down full-time diagnostic data capture"} {"t":{"$date":"2024-08-01T06:47:54.693+00:00"},"s":"I", "c":"CONTROL", "id":20565, "ctx":"initandlisten","msg":"Now exiting"} {"t":{"$date":"2024-08-01T06:47:54.693+00:00"},"s":"I", "c":"CONTROL", "id":23138, "ctx":"initandlisten","msg":"Shutting down","attr":{"exitCode":0}} ## 退出容器 exit从"fix-mongo"导出修复好的数据## 移除修复前的数据,确保已做好数据备份 rm -rf /your/mount/path/mongodata ## 取回修复好的数据 docker cp fix-mongo:/tmp/mongodata /your/mount/path/mongodata ## 重启你的容器 docker restart your_container结束!数据勤备份、数据勤备份、数据勤备份
2024年08月01日
390 阅读
0 评论
0 点赞
2024-06-27
Mongo 关联表查询
摘要Mongo支持与Mysql类似的关联表查询,但比较麻烦。通过与Mysql实现进行对比,便于理解Mongo如何实现。MysqlOrder表关联Product表的id, 实现查询订单支持搜索产品名称。# Order CREATE TABLE `order` ( `id` bigint(20) DEFAULT NULL, `no` varchar(255) DEFAULT NULL, `amount` int(11) DEFAULT NULL COMMENT '单位- 分\n', `productId` bigint(20) DEFAULT NULL, `count` int(11) DEFAULT NULL, `userId` bigint(20) DEFAULT NULL ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4; -- ---------------------------- -- Records of product -- ---------------------------- INSERT INTO `order` (`id`, `no`, `amount`, `productId`, `count`, `userId`) VALUES (1, '2024062701', 10000, 1, 1, 1); INSERT INTO `order` (`id`, `no`, `amount`, `productId`, `count`, `userId`) VALUES (1, '2024062702', 20000, 1, 2, 1); # Product CREATE TABLE `product` ( `id` bigint(20) DEFAULT NULL, `name` varchar(255) DEFAULT NULL, `price` int(11) DEFAULT NULL COMMENT '单位-分' ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4; -- ---------------------------- -- Records of product -- ---------------------------- INSERT INTO `product` (`id`, `name`, `price`) VALUES (1, '测试产品', 10000); 关联查询查询订单、产品名称,通过left join搜索产品名称SELECT o,p.name from `order` as o left JOIN `product` as p on o.productId = p.id where p.name like '%测试%'; ## 结果 #id no amount productId count userId name 1 2024062701 10000 1 1 1 测试产品 1 2024062702 20000 1 2 1 测试产品Mongo同样在mongo中准备Order和Product的集合## insert to product db.product.insertOne({ name: '测试产品', price: 10000 }); ## result { "_id" : ObjectId("667d7fbe04cfea555206351f"), "name" : "测试产品", "price" : 10000 } ## insert to orders db.order.insertMany([ { no: '2024062701', amount: 10000, productId: "667d7fbe04cfea555206351f", count: 1, userId: 1 }, { no: '2024062702', amount: 20000, productId: "667d7fbe04cfea555206351f", count: 2, userId: 1 } ]); # result { "_id" : ObjectId("667d80b804cfea5552063520"), "no" : "2024062701", "amount" : 10000, "productId" : "667d7fbe04cfea555206351f", "count" : 1, "userId" : 1 } { "_id" : ObjectId("667d80b804cfea5552063521"), "no" : "2024062702", "amount" : 20000, "productId" : "667d7fbe04cfea555206351f", "count" : 2, "userId" : 1 }关联查询使用Mongo提供的$lookup来实现关联查询{ "$lookup": { "from": { "connectionName": "<registered-atlas-connection>", "db": "<registered-database-name>", "coll": "<atlas-collection-name>" }, "localField": "<field-in-source-messages>", "foreignField": "<field-in-from-collection>", "as": "<output-array-field>" } }那么我们使用Mongo查询,应该这样写db.order.aggregate( { $lookup: { from: "product", # 需要关联的product表 localField: "proudctId", # order中的productId foreignField: '_id', # 与proudct中的_id关联 as: "p" # product的别名,出现在查询结果中,默认是一个数组 } }, { $unwind: "$p" }, # 将p拉平,如果p是多个,那么会出现多条记录 # 此时得到 order + p:{} { $match: { "p.name": { # 过滤p.name $regex: ".*测试.*", # 使用正则,模糊查询= '%测试%' $options: "i" # 忽略大小写 } } }) 但是,没有查询到结果!?因为 在order中productId是String,而product._id是ObjectId,他俩不是同一个类型,所以是不等的。解决他俩类型不一样,那么把他俩类型弄成一样再比较!db.order.aggregate( { $lookup: { from: "product", let: { pid: "$productId" }, ## 取order.productId到变量pid中 pipeline: [ ## 在product表操作,将pid变成ObjectId后与order._id进行等于比较 { $match: { $expr: { $eq: ["$_id", { $toObjectId: "$$pid" }] } } } ], as: "p" } }, { $unwind: "$p" }, { $match: { "p.name": { $regex: ".*测试.*", $options: "i" } } }) ## Result // collection: order { "_id" : ObjectId("667d80b804cfea5552063520"), "no" : "2024062701", "amount" : 10000, "productId" : "667d7fbe04cfea555206351f", "count" : 1, "userId" : 1, "p" : { "_id" : ObjectId("667d7fbe04cfea555206351f"), "name" : "测试产品", "price" : 10000 } } // collection: order { "_id" : ObjectId("667d80b804cfea5552063521"), "no" : "2024062702", "amount" : 20000, "productId" : "667d7fbe04cfea555206351f", "count" : 2, "userId" : 1, "p" : { "_id" : ObjectId("667d7fbe04cfea555206351f"), "name" : "测试产品", "price" : 10000 } } lookup 解释1、将order.productId声明一个新的变量:pid2、将order._id与 pid比较,此时的pid转换成了obejctIdAI 解释这段代码是MongoDB聚合管道($aggregate)中的一部分,使用了$lookup阶段来实现集合之间的关联查询。这里是将当前集合(假设为订单或其他与产品相关的集合)与"product"集合进行关联,以获取与每个文档中productId字段对应的产品详细信息。我将逐步解释每一部分的含义: $lookup: 这是MongoDB聚合管道的一个操作符,用于在当前集合的文档中连接来自另一个集合的文档。它基于指定的键执行类似SQL中的左外连接。 - from: "product": 指定要连接的另一个集合名为"product"。这是你想要获取额外数据的集合。 - let: { pid: "$productId" }: 定义了一个变量pid,其值为当前文档中的productId字段。这在后续的pipeline中作为查找依据。 - pipeline: 这是一个数组,包含了在"product"集合上执行的聚合管道操作。这里仅有一个操作: - $match: 用于过滤"product"集合中的文档。它只选择那些满足特定条件的文档。 - $expr: 允许在$match中使用聚合表达式。 - $eq: 检查两个表达式是否相等。在这个例子中,检查"product"集合文档的_id字段是否等于通过$toObjectId转换后的$$pid(即来自外部文档的productId,已经转换为ObjectId类型)。 - as: "p": 指定将连接结果存储在当前文档的新字段p中,作为数组形式存在,包含匹配到的所有"product"集合的文档。 Spring Data Mongo Tempalte 实现 Pageable pageable= "Spring data Pageable'; String name= "测试" // Template不支持let,所以自己实现 AggregationOperation l = context -> new Document("$lookup", new Document("from", "product") .append("let", new Document("pid", "$projectId")) .append("pipeline", Arrays.asList( new Document("$match", new Document("$expr", new Document("$eq", Arrays.asList("$_id", new Document("$toObjectId", "$$pid")))) ) )) .append("as", "p")); UnwindOperation unwindOperation = Aggregation.unwind("p"); Criteria c = new Criteria(); if (StringUtils.isNotBlank(name)) { c.and("p.name").regex(".*" + name + ".*", "i"); } Aggregation aggregation = Aggregation.newAggregation( l, unwindOperation, Aggregation.match(c), Aggregation.count().as("c") ); Document count = MongoUtils.template(dbName).aggregate(aggregation, Order.class, Document.class) .getUniqueMappedResult(); // 没有搜索到结果,这里的count会是null if (count == null) { return PageResult.empty(); } Aggregation pageAggregation = Aggregation.newAggregation( // 这有顺序关联: 先跳过,在limit l, unwindOperation, Aggregation.match(c), Aggregation.skip(pageable.getOffset()), Aggregation.limit(pageable.getPageSize()) ); List<Order> data = MongoUtils.template(dbName).aggregate(pageAggregation, Order.class, Order.class) .getMappedResults(); return Page.of(count.getInteger("c"), data);
2024年06月27日
113 阅读
0 评论
0 点赞
2023-11-17
MQTT - LWT(遗愿消息) + Retain 实现设备状态监测
遗愿消息 - LWT(Last Will and Testament)根据MQTT协议,在创建链接时可设置遗愿消息(Will Message)。 它属于链接的一个配置信息,当链接意外断开时将发送该消息。包含但不限于以下情况将发送该信息:服务端发现I/O或网络错误;Client未在Keep Alive time规定时间内进行通信;Client关闭网络链接时未发送0x00代码的“DISCONNECT”包 (非正常关闭);Server 关闭网络链接时未收到0x00代码的“DISCONNECT”包(非正常关闭);遗愿被储存在服务端并与Session进行关联。遗愿消息存储在CONNECT的payload中,由三部分组成:Will Properties : 遗愿属性,一个自定义map。Will Topic : 遗愿主题。Will Payload fields: 遗愿payload,与普通message的payload一样,可自定义。保留消息 -Retain Message保留消息是指发布Message的retain标识为1,当服务端收到一条保留消息:服务端替换该Topic已有的保留消息;订阅者会立即收到消息;未来的订阅者订阅后会立即收到该消息。即Topic只会保存最新的一条保留信息,已订阅的会立即收到,未来订阅的会在订阅时收到该保留消息。清除Topic中的保留消息:如果保留消息的Payload为空(0字节),服务端会该Topic中的保留消息进行移除,并且Topic将不会存储该保留消息。遗愿消息+保留消息实现设备状态监测1、订阅者首先启动,订阅device/status2、设备创建连接时设置保留遗愿消息:{"status":"offline"}3、设备链接成功后,立即发送保留消息:{"status":"online"}此时,订阅者会立即收到消息:{"status":"online"} 设备上线场景1: 订阅者重启1、订阅者会立即收到保留消息:{"status":"online"} 此时设备时在线的。场景2: 设备重启1、设备断开,触发遗愿消息:{"status":"offline"} 设备离线2、设备连接,收到上线的保留消息:{"status":"online"}至此,完成设备状态监测。源码 - JavaDevice:import com.alibaba.fastjson2.JSONObject; import com.hivemq.client.mqtt.MqttClient; import com.hivemq.client.mqtt.MqttClientBuilder; import com.hivemq.client.mqtt.MqttClientState; import com.hivemq.client.mqtt.datatypes.MqttQos; import com.hivemq.client.mqtt.mqtt5.Mqtt5BlockingClient; import com.hivemq.client.mqtt.mqtt5.message.publish.Mqtt5WillPublish; import java.nio.charset.StandardCharsets; import java.util.Map; /** * publish message * * @author chenchen * @Date 2023/11/17 */ public class Publisher { public static void main(String[] args) { MqttClientBuilder clientBuilder = MqttClient.builder() .identifier("cc-status-monitor") .addConnectedListener((context) -> { MqttClientState clientState = context.getClientConfig().getState(); System.out.println(clientState.isConnected()); }) .addDisconnectedListener(context -> System.out.println(context.getCause().getMessage())) .serverHost("broker.hivemq.com") .serverPort(1883); // will message with retain Mqtt5WillPublish offline = Mqtt5WillPublish.builder() .topic("device/status") .qos(MqttQos.AT_LEAST_ONCE) .retain(false) .payload(JSONObject.toJSONString(Map.of("status", "offline")).getBytes(StandardCharsets.UTF_8)) .build(); // set will message when create connect Mqtt5BlockingClient client = clientBuilder.useMqttVersion5() .willPublish(offline) .buildBlocking(); // connect client.connect(); // success connect, publish a retain massage of online if (client.getState().isConnected()) { Mqtt5WillPublish online = Mqtt5WillPublish.builder() .topic("device/status") .qos(MqttQos.AT_LEAST_ONCE) .retain(true) .payload(JSONObject.toJSONString(Map.of("status", "online")).getBytes(StandardCharsets.UTF_8)) .build(); client.publish(online); } } } Subscribe:import cn.hutool.core.util.StrUtil; import com.hivemq.client.mqtt.MqttClient; import com.hivemq.client.mqtt.MqttClientState; import com.hivemq.client.mqtt.mqtt5.Mqtt5AsyncClient; import com.hivemq.client.mqtt.mqtt5.message.subscribe.Mqtt5Subscribe; import com.hivemq.client.mqtt.mqtt5.message.subscribe.Mqtt5Subscription; import java.nio.ByteBuffer; import java.nio.charset.StandardCharsets; import java.util.Optional; /** * subscribe * @author chenchen * @Date 2023/11/17 */ public class Subscriber { public static void main(String[] args) { Mqtt5AsyncClient client = MqttClient.builder() .addConnectedListener((context) -> { MqttClientState clientState = context.getClientConfig().getState(); System.out.println(clientState.isConnected()); }) .addDisconnectedListener(context -> System.out.println(context.getCause().getMessage())) .serverHost("broker.hivemq.com") .serverPort(1883) .useMqttVersion5() .buildAsync(); // subscribe topic Mqtt5Subscribe subscribe = Mqtt5Subscribe.builder() .addSubscription(Mqtt5Subscription.builder() .topicFilter("device/status") .build()) .build(); // connect client.connect(); // handle message client.subscribe(subscribe, mqtt5Publish -> { Optional<ByteBuffer> payload = mqtt5Publish.getPayload(); payload.ifPresent(byteBuffer -> System.out.println("receive message with payload:" + StrUtil.str(byteBuffer, StandardCharsets.UTF_8))); }); } }引用《物联网系统开发:从0到1构建IoT平台》付强Will : https://docs.oasis-open.org/mqtt/mqtt/v5.0/os/mqtt-v5.0-os.html#_Toc3901042Return: https://docs.oasis-open.org/mqtt/mqtt/v5.0/os/mqtt-v5.0-os.html#_Toc3901104hivemq : https://github.com/hivemq/hivemq-mqtt-clientMqtt Server: broker.hivemq.com
2023年11月17日
402 阅读
0 评论
0 点赞
2023-09-24
摘自— 《纳瓦尔宝典》
“我认为,过去就是过去了,没有回忆,没有遗憾,没有放不下的人,没有忘不掉的旅行。既往不恋。人之所以感到痛苦,很多时候是因为拿以往和现在做比较。”
2023年09月24日
189 阅读
0 评论
0 点赞
2023-06-21
PromptAI - 开启交互式AI仅一步之遥!
背景我们一直从事企业服务,早在2020年通过服务国内的客户,我们就敏感的发现交互技术的变革即将发生,新的交互时代即将来临。我们把自身的专业知识、经验和能力集成到了我们的产品中,希望能帮助客户获得相应的能力,能更好的面对这个伟大的时代!PromptAI为快速对话设计和部署⽽构建的⽆代码开发环境。它基于RASA,但⼜不需要特定的RASA 知识来设计和部署聊天机器⼈。使⽤⾃然语⾔来设计⾃然语⾔对话,它最⼤限度地减少了标注和编码⼯作,旨在⼏分钟内发布企业级聊天机器⼈。有哪些特点?使用友好无需编程, 近似自然语言的方式编辑对话,能写word就能上手免费且拥有所有功能,可上线两个独立的chatbot [真-无任何费用]解决80%以上的对话需求,24小时在线服务开发友好:Web无侵入式部署,移动端完美适配FAQ、多轮、变量提取、Form、Webhook、Goto...免费下载⽣成的RASA源码ChatGPT支持私有部署 [免费版已推出,欢迎联系]适合那些用户?个人: 对chatbot/Rasa感兴趣个人开发者: 提供免费机器人服务,无任何运行及运维成本,一行代码无侵入式集成到您的应用中/小企业用户: 无需专门组建客服团队,已有客户团队的情况下极大降低客服团队工作量。其他: 欢迎联系我们获取专属解决方案.预览近似自然语言的方式编辑对话一键发布,无侵入式部署完美兼容移动端如何注册?https://www.promptai.cn联系我们:email: info@promptai.cnWechat: 官网左下角微信扫码加入群聊我们把我们产品最基本的部分贡献出来,方便企业用户开发、设计交互式对话系统,并在这个基础之上,我们提供了许多针对企业的咨询服务和解决方案。如果想更多的了解我们的产品和服务,请访问我们的网站,期待你的光临!
2023年06月21日
207 阅读
0 评论
1 点赞
1
2
...
4