Ceph OSD处理客户端写操作处理流程的示例分析

发布时间:2021-12-17 10:11:26 作者:小新
来源:亿速云 阅读:161

这篇文章主要介绍Ceph OSD处理客户端写操作处理流程的示例分析,文中介绍的非常详细,具有一定的参考价值,感兴趣的小伙伴们一定要看完!

1、OSD从client端收到请求的处理流程

OSD::ms_fast_dispatch()

|__OSD::dispatch_session_waiting()

     |__OSD::dispatch_op_fast()

          |__OSD::handle_op()

               |__OSD::get_pg_or_queue_for_pg()

               |__OSD::enqueue_op()

                    |__PG::queue_op()

                         |__OSD::op_wq.queue()          将client的请求写入到队列

主OSD处理从client端请求的处理流程

OSD::ShardOpWQ::_process()

|__OSD::dequeue_op()

     |__ReplicatedPG::do_request()

          |__ReplicatedPG::do_op()

               |__ReplicatedPG::execute_ctx()

                    |__ReplicatedPG::preapare_transcation()

                         |__ReplicatedPG::do_osd_ops()

                    |__ReplicatedPG::issue_repop()

                         |__ReplicateBackend::submit_transaction()

                              |__ReplicateBackend::issue_op()                    主OSD将写请求发送到从OSDs

                              |__ReplicatedPG::queue_transcations()          写主OSD FileJournal

                                   |__ObjectStore::queue_transactions()

                                        |__FileStore::queue_transactions()

                                             |__JournalingObjectStore::_op_journal_transactions()

                                                  |__FileJournal::submit_entry()     将日志写入到FileJournal的writeq队列

                    |__ReplicatedPG::eval_repop()

主OSD的FileJournal写线程

FileJournal::write_thread_entry()           负责从writeq队列中获取待写入的日志

|__FileJournal::prepare_multi_write()     构成日志

|__FileJournal::do_write()                       实际写日志

     |__FIleJournal::queue_completions_thru()

          |__completion_peek_front()          获取C_JournalAhead类实例

          |__finisher->queue()

          |__finisher_cond.Signal()               启动FileJournal的finisher线程处理C_JournalAhead类实例

主OSD的C_JournalAhead类处理

C_JournalAhead::finish()

|__FileStore::_journaled_ahead()

     |__FileStore::queue_op()

          |__FileStore::op_wq.queue()          将数据写入到FileStore写线程队列中

     |__ondisk_finisher.queue(ondisk)     调用ReplicatedBackend->op_commit()回调函数

主OSD的FileStore写数据线程类处理

FileStore::op_wq

|__FileStore::op_wq._process()

     |__FileStore::_do_op()

          |__FileStore::_do_transactions()

               |__FileStore::_do_transaction()

                    |__FileStore::_write()          落盘到FileStore中(写入指定的文件中)

|__FileStore::op_wq._process_finish()

     |__FileStore::_finish_op()

          |__op_finisher.queue()                    调用ReplicatedPG::op_applied()回调函数


2、从OSD处理主OSD端发送过来的请求

OSD::ms_fast_dispatch()

|__OSD::dispatch_session_waiting()

     |__OSD::dispatch_op_fast()

          |__OSD::handle_replica_op()

               |__OSD::get_pg_or_queue_for_pg()

               |__OSD::enqueue_op()

                    |__PG::queue_op()

                         |__OSD::op_wq.queue()          将主OSD的请求写入到从OSD处理队列中

从OSD处理workqueue

OSD::ShardOpWQ::_process()

|__OSD::dequeue_op()

     |__ReplicatedPG::do_request()

          |__ReplicatedBackend::handle_message()

               |__ReplicatedBackend::sub_op_modify()

                    |__ReplicatedBackend::sub_op_modify_impl()

                         |__注册commit回调函数类C_OSD_RepModifyCommit()

                         |__注册Apply回调函数类C_OSD_RepModifyApply()

                         |__ReplicatedPG::queue_transcations()          从OSD写FileJournal

                                   |__ObjectStore::queue_transactions()

                                        |__FileStore::queue_transactions()

                                             |__JournalingObjectStore::_op_journal_transactions()

                                                  |__FileJournal::submit_entry()     将日志写入到FileJournal的writeq队列

                    |__ReplicatedPG::eval_repop()

从OSD的FileJournal写线程

FileJournal::write_thread_entry()           负责从writeq队列中获取待写入的日志

|__FileJournal::prepare_multi_write()     构成日志

|__FileJournal::do_write()                       实际写日志

     |__FIleJournal::queue_completions_thru()

          |__completion_peek_front()          获取C_JournalAhead类实例

          |__finisher->queue()

          |__finisher_cond.Signal()               启动FileJournal的finisher线程处理C_JournalAhead类实例

从OSD的C_JournalAhead类处理

C_JournalAhead::finish()

|__FileStore::_journaled_ahead()

     |__FileStore::queue_op()

          |__FileStore::op_wq.queue()          将数据写入到FileStore写线程队列中

     |__ondisk_finisher.queue(ondisk)     调用ReplicatedBackend->op_commit()回调函数

从OSD的日志写完成后的处理

C_OSD_RepModifyCommit()

|__finish()

     |__ReplicatedBackend::sub_op_modify_commit()

          |__生成MOSDRepOpReply消息                                   CEPH_OSD_FLAG_ONDISK

          |__ReplicatedPG::send_message_osd_cluster()          将从OSD回复信息发给主OSD

               |__OSD::send_message_osd_cluster()

从OSD的FileStore写数据线程类处理

FileStore::op_wq

|__FileStore::op_wq._process()

     |__FileStore::_do_op()

          |__FileStore::_do_transactions()

               |__FileStore::_do_transaction()

                    |__FileStore::_write()          落盘到FileStore中(写入指定的文件中)

|__FileStore::op_wq._process_finish()

     |__FileStore::_finish_op()

          |__op_finisher.queue()                    调用ReplicatedPG::op_applied()回调函数

从OSD的落盘完成后的处理

C_OSD_RepModifyApply()

|__finish()

     |__ReplicatedBackend::sub_op_modify_applied()

          |__生成MOSDRepOpReply消息                                   CEPH_OSD_FLAG_ACK

          |__ReplicatedPG::send_message_osd_cluster()          将从OSD回复信息发给主OSD

               |__OSD::send_message_osd_cluster()


3、主OSD处理从OSD发送Reply处理流程

从OSD处理主OSD端发送过来的请求

OSD::ms_fast_dispatch()

|__OSD::dispatch_session_waiting()

     |__OSD::dispatch_op_fast()

          |__OSD::handle_replica_op()

               |__OSD::get_pg_or_queue_for_pg()

               |__OSD::enqueue_op()

                    |__PG::queue_op()

                         |__OSD::op_wq.queue()          将从OSD的reply写入到主OSD处理队列中

主OSD处理从OSD端发送过来的reply

OSD::ShardOpWQ::_process()

|__OSD::dequeue_op()

     |__ReplicatedPG::do_request()

          |__ReplicatedBackend::handle_message()

               |__ReplicatedBackend::sub_op_modify_reply()

                    |__ip_op.waiting_for_commit.erase()

                    |__ip_op.waiting_for_applied.erase()

                    |__对于ip_op.waiting_for_applied为空,则调用ip_op.on_applied->complete(0),即:调用on_all_applied回调函数处理。on_all_applied回调函数在ReplicatedPG.cc::execute_ctx()函数中注册,用来对所有副本都落盘完成后的处理。on_all_applied()函数创建一个MOSDOpReply消息并且设置消息的flags=CEPH_OSD_FLAG_ACK且将该消息发送给客户端;

                    |__对于ip_op.waiting_for_commit为空,则调用ip_op.on_commit->complete(0),即:调用on_all_commit回调函数处理。on_all_commit回调函数在ReplicatedPG.cc::execute_ctx()函数中注册,用来对所有副本在写FileJournal完成后的处理。on_all_commit()函数创建一个MOSDOpReply消息并且设置消息的flags=CEPH_OSD_FLAG_ACK|CEPH_OSD_FLAG_ONDISK且将该消息发送给客户端;


以上是“Ceph OSD处理客户端写操作处理流程的示例分析”这篇文章的所有内容,感谢各位的阅读!希望分享的内容对大家有帮助,更多相关知识,欢迎关注亿速云行业资讯频道!

推荐阅读:
  1. Ceph架构和组件
  2. Ceph实现读写流程的示例分析

免责声明:本站发布的内容(图片、视频和文字)以原创、转载和分享为主,文章观点不代表本网站立场,如果涉及侵权请联系站长邮箱:is@yisu.com进行举报,并提供相关证据,一经查实,将立刻删除涉嫌侵权内容。

ceph osd

上一篇:Linux面试题有哪些

下一篇:python匿名函数怎么创建

相关阅读

您好,登录后才能下订单哦!

密码登录
登录注册
其他方式登录
点击 登录注册 即表示同意《亿速云用户服务条款》