搜索到
3
篇与
docker
的结果
-
Mongo错误修复 - WiredTiger metadata corruption detected 免责声明本文提供的所有信息仅供参考，作者不对使用本文所含信息而导致的任何直接或间接损失负责。在进行MongoDB数据恢复操作之前，请务必备份您的数据并仔细阅读MongoDB官方文档。数据恢复操作具有风险，可能导致数据的永久丢失或损坏。作者建议在执行任何数据恢复步骤之前，先在测试环境中进行尝试，并咨询有经验的数据库管理员或技术专家。背景系统: macOS 13.6Mongo: 5.0.3 (Docker 容器部署)原因: 由于Docker Desktop在Mac上运行消耗资源非常高，所以又安装了OrbStack进行docker管理。 在OrbStack运行时，意外的打开了Docker Desktop（噩耗的开始）。 刚开始没在意，中午吃饭发现系统刷新失败了！？ 果然还是出问题了。782c0bd0dfb4 mongo:5.0.3 "docker-entrypoint.s…" 6 weeks ago Restarting (14) 46 seconds ago mongodbmongodb的容器一直在重启，于是看看log:{"t":{"$date":"2024-08-01T06:15:22.352+00:00"},"s":"I", "c":"NETWORK", "id":4915701, "ctx":"-","msg":"Initialized wire specification","attr":{"spec":{"incomingExternalClient":{"minWireVersion":0,"maxWireVersion":13},"incomingInternalClient":{"minWireVersion":0,"maxWireVersion":13},"outgoing":{"minWireVersion":0,"maxWireVersion":13},"isInternalClient":true}}} {"t":{"$date":"2024-08-01T06:15:22.352+00:00"},"s":"I", "c":"CONTROL", "id":23285, "ctx":"-","msg":"Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'"} {"t":{"$date":"2024-08-01T06:15:22.354+00:00"},"s":"W", "c":"ASIO", "id":22601, "ctx":"main","msg":"No TransportLayer configured during NetworkInterface startup"} {"t":{"$date":"2024-08-01T06:15:22.355+00:00"},"s":"I", "c":"NETWORK", "id":4648601, "ctx":"main","msg":"Implicit TCP FastOpen unavailable. If TCP FastOpen is required, set tcpFastOpenServer, tcpFastOpenClient, and tcpFastOpenQueueSize."} {"t":{"$date":"2024-08-01T06:15:22.355+00:00"},"s":"W", "c":"ASIO", "id":22601, "ctx":"main","msg":"No TransportLayer configured during NetworkInterface startup"} {"t":{"$date":"2024-08-01T06:15:22.355+00:00"},"s":"I", "c":"REPL", "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"TenantMigrationDonorService","ns":"config.tenantMigrationDonors"}} {"t":{"$date":"2024-08-01T06:15:22.355+00:00"},"s":"I", "c":"REPL", "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"TenantMigrationRecipientService","ns":"config.tenantMigrationRecipients"}} {"t":{"$date":"2024-08-01T06:15:22.356+00:00"},"s":"I", "c":"CONTROL", "id":4615611, "ctx":"initandlisten","msg":"MongoDB starting","attr":{"pid":1,"port":27017,"dbPath":"/data/db","architecture":"64-bit","host":"2f51c27fb1c7"}} {"t":{"$date":"2024-08-01T06:15:22.356+00:00"},"s":"I", "c":"CONTROL", "id":23403, "ctx":"initandlisten","msg":"Build Info","attr":{"buildInfo":{"version":"5.0.3","gitVersion":"657fea5a61a74d7a79df7aff8e4bcf0bc742b748","openSSLVersion":"OpenSSL 1.1.1f 31 Mar 2020","modules":[],"allocator":"tcmalloc","environment":{"distmod":"ubuntu2004","distarch":"x86_64","target_arch":"x86_64"}}}} {"t":{"$date":"2024-08-01T06:15:22.356+00:00"},"s":"I", "c":"CONTROL", "id":51765, "ctx":"initandlisten","msg":"Operating System","attr":{"os":{"name":"Ubuntu","version":"20.04"}}} {"t":{"$date":"2024-08-01T06:15:22.356+00:00"},"s":"I", "c":"CONTROL", "id":21951, "ctx":"initandlisten","msg":"Options set by command line","attr":{"options":{"net":{"bindIp":"*"}}}} {"t":{"$date":"2024-08-01T06:15:22.357+00:00"},"s":"I", "c":"STORAGE", "id":22270, "ctx":"initandlisten","msg":"Storage engine to use detected by data files","attr":{"dbpath":"/data/db","storageEngine":"wiredTiger"}} {"t":{"$date":"2024-08-01T06:15:22.357+00:00"},"s":"I", "c":"STORAGE", "id":22315, "ctx":"initandlisten","msg":"Opening WiredTiger","attr":{"config":"create,cache_size=3455M,session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),builtin_extension_config=(zstd=(compression_level=6)),file_manager=(close_idle_time=600,close_scan_interval=10,close_handle_minimum=250),statistics_log=(wait=0),verbose=[recovery_progress,checkpoint_progress,compact_progress],"}} {"t":{"$date":"2024-08-01T06:15:22.776+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1722492922:776094][1:0x7f18c7605c80], file:WiredTiger.wt, connection: __wt_btree_tree_open, 637: unable to read root page from file:WiredTiger.wt: WT_ERROR: non-specific WiredTiger error"}} {"t":{"$date":"2024-08-01T06:15:22.776+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1722492922:776189][1:0x7f18c7605c80], file:WiredTiger.wt, connection: __wt_btree_tree_open, 643: WiredTiger has failed to open its metadata: WT_ERROR: non-specific WiredTiger error"}} {"t":{"$date":"2024-08-01T06:15:22.776+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1722492922:776210][1:0x7f18c7605c80], file:WiredTiger.wt, connection: __wt_btree_tree_open, 646: This may be due to the database files being encrypted, being from an older version or due to corruption on disk: WT_ERROR: non-specific WiredTiger error"}} {"t":{"$date":"2024-08-01T06:15:22.776+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1722492922:776232][1:0x7f18c7605c80], file:WiredTiger.wt, connection: __wt_btree_tree_open, 649: You should confirm that you have opened the database with the correct options including all encryption and compression options: WT_ERROR: non-specific WiredTiger error"}} {"t":{"$date":"2024-08-01T06:15:22.783+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1722492922:783621][1:0x7f18c7605c80], file:WiredTiger.wt, connection: __wt_btree_tree_open, 637: unable to read root page from file:WiredTiger.wt: WT_ERROR: non-specific WiredTiger error"}} {"t":{"$date":"2024-08-01T06:15:22.783+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1722492922:783671][1:0x7f18c7605c80], file:WiredTiger.wt, connection: __wt_btree_tree_open, 643: WiredTiger has failed to open its metadata: WT_ERROR: non-specific WiredTiger error"}} {"t":{"$date":"2024-08-01T06:15:22.783+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1722492922:783683][1:0x7f18c7605c80], file:WiredTiger.wt, connection: __wt_btree_tree_open, 646: This may be due to the database files being encrypted, being from an older version or due to corruption on disk: WT_ERROR: non-specific WiredTiger error"}} {"t":{"$date":"2024-08-01T06:15:22.783+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1722492922:783688][1:0x7f18c7605c80], file:WiredTiger.wt, connection: __wt_btree_tree_open, 649: You should confirm that you have opened the database with the correct options including all encryption and compression options: WT_ERROR: non-specific WiredTiger error"}} {"t":{"$date":"2024-08-01T06:15:22.790+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1722492922:790000][1:0x7f18c7605c80], file:WiredTiger.wt, connection: __wt_btree_tree_open, 637: unable to read root page from file:WiredTiger.wt: WT_ERROR: non-specific WiredTiger error"}} {"t":{"$date":"2024-08-01T06:15:22.790+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1722492922:790045][1:0x7f18c7605c80], file:WiredTiger.wt, connection: __wt_btree_tree_open, 643: WiredTiger has failed to open its metadata: WT_ERROR: non-specific WiredTiger error"}} {"t":{"$date":"2024-08-01T06:15:22.790+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1722492922:790057][1:0x7f18c7605c80], file:WiredTiger.wt, connection: __wt_btree_tree_open, 646: This may be due to the database files being encrypted, being from an older version or due to corruption on disk: WT_ERROR: non-specific WiredTiger error"}} {"t":{"$date":"2024-08-01T06:15:22.790+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1722492922:790063][1:0x7f18c7605c80], file:WiredTiger.wt, connection: __wt_btree_tree_open, 649: You should confirm that you have opened the database with the correct options including all encryption and compression options: WT_ERROR: non-specific WiredTiger error"}} {"t":{"$date":"2024-08-01T06:15:22.791+00:00"},"s":"W", "c":"STORAGE", "id":22347, "ctx":"initandlisten","msg":"Failed to start up WiredTiger under any compatibility version. This may be due to an unsupported upgrade or downgrade."} {"t":{"$date":"2024-08-01T06:15:22.791+00:00"},"s":"W", "c":"STORAGE", "id":22348, "ctx":"initandlisten","msg":"WiredTiger metadata corruption detected"} {"t":{"$date":"2024-08-01T06:15:22.791+00:00"},"s":"F", "c":"STORAGE", "id":50944, "ctx":"initandlisten","msg":"Please read the documentation for starting MongoDB with --repair here: http://dochub.mongodb.org/core/repair"} {"t":{"$date":"2024-08-01T06:15:22.791+00:00"},"s":"F", "c":"-", "id":23091, "ctx":"initandlisten","msg":"Fatal assertion","attr":{"msgid":50944,"file":"src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp","line":685}} {"t":{"$date":"2024-08-01T06:15:22.791+00:00"},"s":"F", "c":"-", "id":23092, "ctx":"initandlisten","msg":"\n\n***aborting after fassert() failure\n\n"} 关键信息： Failed to start up WiredTiger under any compatibility version. This may be due to an unsupported upgrade or downgrade. WiredTiger metadata corruption detected Please read the documentation for starting MongoDB with --repair here: http://dochub.mongodb.org/core/repair1、"WiredTiger metadata corruption detected" 原因清楚了，存储引擎的元数据坏掉了。在 MongoDB 中，WiredTiger 是一个用于存储引擎的名字。WiredTiger 引擎提供了高效的数据压缩和并发处理能力，从 MongoDB 3.2 版本开始成为默认的存储引擎。WiredTiger 文件实际上包含了数据文件和索引文件。下面是一些有关 WiredTiger 文件的关键点： 存储引擎：WiredTiger 是 MongoDB 的存储引擎，它负责管理数据库的数据存储、索引、并发控制和缓存等方面。 数据文件：在 MongoDB 数据库目录中，WiredTiger 存储引擎会创建一系列文件，用于存储数据和索引。这些文件包括 .wt 后缀的文件。 配置文件：WiredTiger.wt 是一个主要的配置文件，它包含了有关存储引擎配置和元数据的信息。 日志文件：WiredTiger 还使用一组日志文件（如 WiredTigerLog 开头的文件）来进行写入操作的日志记录，以便在发生崩溃时进行数据恢复。 压缩和并发：WiredTiger 提供了数据压缩功能，可以显著减少存储空间需求。此外，它还支持多线程并发写入，从而提高数据库性能。 简而言之，WiredTiger 是 MongoDB 用于管理和存储数据的核心组件，提供了高效的数据存储和管理能力。2、"http://dochub.mongodb.org/core/repair"问题发生时系统没有数据写入，所以我们数据应该没问题，根据提示我们进行修复:mongod --dbpath /data/db --repair修复由于在容器中部署的Mongo，不可能在这个容器中进行修复，所以重新起一个同版本的容器进行修复。步骤：备份数据确定Mongo 版本启动修复Mongo的容器"fix-mongo"拷贝数据到"fix-mongo"并修复从"fix-mongo"导出修复好的数据数据备份zip -r bad_mongodata_yyyyMMdd.zip /your/mount/path/mongodata确定Mongo版本 docker inspect your_container |grep 'MONGO_VERSION' "MONGO_VERSION=5.0.3"我这里是"5.0.3"启动修复Mongo的容器"fix-mongo"docker run --name fix-mongo -d mongo:your_mongo_version拷贝数据到"fix-mongo"并修复将整个数据目录拷贝到"fix-mongo"的"/tmp"## 拷贝数据到容器 docker cp /your/mount/path/mongodata fix-mongo:/tmp ## 修复 - 等待结束 mongod --dbpath /tmp/mongodata --repair ## 修复的部分log ... {"t":{"$date":"2024-08-01T06:47:54.435+00:00"},"s":"I", "c":"STORAGE", "id":22319, "ctx":"initandlisten","msg":"Finished shutting down session sweeper thread"} {"t":{"$date":"2024-08-01T06:47:54.451+00:00"},"s":"I", "c":"STORAGE", "id":4795902, "ctx":"initandlisten","msg":"Closing WiredTiger","attr":{"closeConfig":"leak_memory=true,"}} {"t":{"$date":"2024-08-01T06:47:54.693+00:00"},"s":"I", "c":"STORAGE", "id":4795901, "ctx":"initandlisten","msg":"WiredTiger closed","attr":{"durationMillis":242}} {"t":{"$date":"2024-08-01T06:47:54.693+00:00"},"s":"I", "c":"STORAGE", "id":22279, "ctx":"initandlisten","msg":"shutdown: removing fs lock..."} {"t":{"$date":"2024-08-01T06:47:54.693+00:00"},"s":"I", "c":"-", "id":4784931, "ctx":"initandlisten","msg":"Dropping the scope cache for shutdown"} {"t":{"$date":"2024-08-01T06:47:54.693+00:00"},"s":"I", "c":"FTDC", "id":20626, "ctx":"initandlisten","msg":"Shutting down full-time diagnostic data capture"} {"t":{"$date":"2024-08-01T06:47:54.693+00:00"},"s":"I", "c":"CONTROL", "id":20565, "ctx":"initandlisten","msg":"Now exiting"} {"t":{"$date":"2024-08-01T06:47:54.693+00:00"},"s":"I", "c":"CONTROL", "id":23138, "ctx":"initandlisten","msg":"Shutting down","attr":{"exitCode":0}} ## 退出容器 exit从"fix-mongo"导出修复好的数据## 移除修复前的数据，确保已做好数据备份 rm -rf /your/mount/path/mongodata ## 取回修复好的数据 docker cp fix-mongo:/tmp/mongodata /your/mount/path/mongodata ## 重启你的容器 docker restart your_container结束！数据勤备份、数据勤备份、数据勤备份 -
基于K8s + Jenkins+Docker + Gitee 自动部署 - 配置 Jenkins Item + Gitee Webhook (二) 继续上一篇我们在k8s中搭建了Jenkins环境，接下来我们分别在Jenkins和Gitee中完善配置Gitee 引导配置：https://gitee.com/help/articles/4193Jenkins一、安装Gitee插件这里已经安装好了，具体步骤可参考：https://gitee.com/help/articles/4193#article-header3接下来，需要配置插件:https://gitee.com/help/articles/4193#article-header4这里出现成功即可二、创建构建任务这里我们期望push到develop分支时，能触发后端自动打包部署。1)、在Jenkins首页左上角，点击"New Item",输入名称2)、这里我们选择"Freestyle project"3)、"General"部分"Gitee 链接"默认配置好了4)、"Source Code Management" 配置源码管理：仓库地址、用户登录凭证仓库地址可以在仓库的“克隆”中找到 ：如https://gitee.com/xx/xx.git在"Credentials"处点击"Add"配置Gitee账号用户名/密码5)、配置需要打包的分支，这里我们填写develop6)、配置"Build Triggers",即在什么情况可触发打包部署。这里我们勾选"Gitee webhook 触发构建.."展开后，继续勾选"推送代码"这里记录下"需要在 Gitee webhook 中填写 URL:..."，这里需要配置公网可访问的ip或域名7)、接下来配置"构建指令过滤":这里我们只允许develop才能触发8)、生产一个固定的密码，用于Webhook请求鉴权9)、"Build Environment"配置收Webhook时执行的操作，这里我们选择执行脚本点击"Add build step"选择"Execnte shell"运行脚本"sh restarxx.sh"这里的 restarxx.sh 在仓库的根目录中该脚本执行：打包jar、build docker image 、重启container9)、点击"save"进行保存Gitee1)、进入到仓库配置，新增WebhookUrl:参考Jenkins步骤6给出的URL,这里需要转换公网可访问的ip+portWebHook 密码/签名密钥:参考 Jenkins步骤8生成的密码2)、验证Webhook配置好后可进行"测试",提示返回200即可push 到develop分支在Jenkins中可看到构建情况
-
基于K8s + Jenkins+Docker + Gitee 自动部署 - 在K8s中安装Jenkins (一) 背景目前在开发过程中，每次更新测试环境需要手动执行一个脚本：pull最新的代码、build jar、build docker image最后再基于最新的docker image启动容器。于是想“偷个懒”，将这些步骤自动化起来:当指定分支收到push后，触发打包、重启docker container.环境服务器: k8s集群，有N个node仓库: Gitee部署: 部署在集群中的某个node中一、创建namespacekubectl create ns jenkins二、创建Jenkins Deployment & Service准备配置文件：jenkins.yamlapiVersion: apps/v1 kind: Deployment metadata: namespace: jenkins name: jenkins-deployment spec: replicas: 1 selector: matchLabels: app: jenkins template: metadata: labels: app: jenkins spec: nodeName: Your_node_name # 可选，如果不挂载到本地目录，参考文末引用文章 containers: - name: jenkins image: jenkinsci/blueocean:latest ports: - containerPort: 8080 volumeMounts: - name: jenkins-home mountPath: /var/jenkins_home - name: docker-sock mountPath: /var/run/docker.sock volumes: - name: jenkins-home hostPath: path: /your/local/file/path # update me type: Directory - name: docker-sock # 声明挂载docker hostPath: path: /var/run/docker.sock type: Socket --- apiVersion: v1 kind: Service metadata: namespace: jenkins name: jenkins spec: type: NodePort ports: - port: 8080 targetPort: 8080 nodePort: 30005 # 在本机上暴露Jenkins对外服务端口 selector: app: jenkins 创建 Deployment & Service:kubectl create -f jenkins.yaml 注：这里准备把Jenkins的数据映射到host目录，所以使用hostPath挂载/var/jenkins_home，于是在指定的node上运行pod由于项目需要打包用到docker,所以将/var/run/docker.sock通过Socket方式挂载到容器（docker in docker）在Jenkins的pod中执行docker ps -a遇到/var/run/docker.sock: connect: permission denied错误，此时回到host执行：chmod 0777 /var/run/docker.sock赋予权限三、登录配置使用浏览器打开：http://your_k8s_node_ip:port 如:http://127.0.0.1:30005这里要求输入初始密码:登录pod的容器，查看对应的密码。图片中告知的路径：/var/lib/jenkins/secrets/initialAdminPassword # 获取node 名称 [root@cc jenkins]# kubectl get po -n jenkins NAME READY STATUS RESTARTS AGE jenkins-deployment-5db55f9db-7mrk2 1/1 Running 4 4d23h # 进入pod [root@cc jenkins]# kubectl exec -it jenkins-deployment-5db55f9db-7mrk2 -c jenkins /bin/bash -n jenkins # cat 密码 [root@cc jenkins]# cat /var/lib/jenkins/secrets/initialAdminPassword ****** 最后，按照推荐进行安装插件即可。参考：1、基于k8s安装Jenkinshttps://blog.csdn.net/qq_38942551/article/details/107769120
