{"guid":"2dd3d338-37ae-5b15-8292-1f124c0235a4","title":"Traceloop for systemd and Kubernetes + Inspektor Gadget","subtitle":null,"slug":"ASG2019-159-traceloop-for-systemd-and-kubernetes-inspektor-gadget","link":"https://cfp.all-systems-go.io/ASG2019/talk/98A9LW/","description":"Presenting [traceloop](https://github.com/kinvolk/traceloop), a “time travel” tracing tool to trace system calls in cgroups using BPF and overwritable ring buffers.\n\nMany people use the “strace” tool to synchronously trace system calls using ptrace. [Traceloop](https://github.com/kinvolk/traceloop) similarly traces system calls but asynchronously in the background, using BPF and tracing per cgroup. I’ll show how it can be integrated with systemd and with Kubernetes via [Inspektor Gadget](https://github.com/kinvolk/inspektor-gadget).\n\nTraceloop's traces are recorded in a fast, in-memory, overwritable ring buffer like a flight recorder. As opposed to “strace”, the tracing could be permanently enabled on systemd services or Kubernetes pods and inspected in case of a crash. This is like a always-on “strace in the past”.\n\nTraceloop uses BPF through the gobpf library. Several new features have been added in gobpf for the needs of traceloop: support for overwritable ring buffers and swapping buffers when the userspace utility dumps the buffer.\n\nhttps://github.com/kinvolk/traceloop\nhttps://github.com/kinvolk/inspektor-gadget\nhttps://github.com/iovisor/gobpf\n\nSlides: https://docs.google.com/presentation/d/1zIZUrTrD7FkS9pHnWz87ZmoLTrO1g9-J_lDMD7E5kdo/edit","original_language":"eng","persons":["Alban Crequy"],"tags":["asg2019","159","2019"],"view_count":96,"promoted":false,"date":"2019-09-20T00:00:00.000+02:00","release_date":"2019-09-21T02:00:00.000+02:00","updated_at":"2026-02-08T12:30:06.711+01:00","length":2250,"duration":2250,"thumb_url":"https://static.media.ccc.de/media/events/all_systems_go/2019/159-hd.jpg","poster_url":"https://static.media.ccc.de/media/events/all_systems_go/2019/159-hd_preview.jpg","timeline_url":"https://static.media.ccc.de/media/events/all_systems_go/2019/159-hd.timeline.jpg","thumbnails_url":"https://static.media.ccc.de/media/events/all_systems_go/2019/159-hd.thumbnails.vtt","frontend_link":"https://media.ccc.de/v/ASG2019-159-traceloop-for-systemd-and-kubernetes-inspektor-gadget","url":"https://api.media.ccc.de/public/events/2dd3d338-37ae-5b15-8292-1f124c0235a4","conference_title":"All Systems Go! 2019","conference_url":"https://api.media.ccc.de/public/conferences/asg2019","related":[{"event_id":7864,"event_guid":"3d439f1d-e67a-5e92-96b6-3a9c5ad30965","weight":4},{"event_id":7865,"event_guid":"9dbd53a6-97d6-55b6-8ef0-e339aae4ff20","weight":4},{"event_id":7874,"event_guid":"1ea14c32-c58c-513c-ab1f-40ac082e985c","weight":1},{"event_id":7875,"event_guid":"21aca390-bc0c-5eef-a867-1b57a7ee36ab","weight":4},{"event_id":7877,"event_guid":"0ca00408-9684-58d3-a7c3-7d7b6cae6b75","weight":5},{"event_id":7888,"event_guid":"dad4ba00-7141-5e7d-af7d-02f492f4b5e5","weight":6},{"event_id":7890,"event_guid":"8a203dd4-c4d8-51af-b01e-8a199a515c16","weight":3},{"event_id":7894,"event_guid":"b9883475-56ad-5749-9c65-e178a1e6bbfb","weight":7},{"event_id":7910,"event_guid":"b0b3fa50-d9db-511a-9fb1-9edb96bd3401","weight":5},{"event_id":7922,"event_guid":"090131ec-ac5c-5b60-85a0-6d080ea4054c","weight":3},{"event_id":7926,"event_guid":"47ead120-c630-5091-bf6e-5b72c6f98ec8","weight":5}],"recordings":[{"size":176,"length":2250,"mime_type":"video/mp4","language":"eng","filename":"asg2019-159-eng-Traceloop_for_systemd_and_Kubernetes_Inspektor_Gadget_hd.mp4","state":"new","folder":"h264-hd","high_quality":true,"width":1920,"height":1080,"updated_at":"2019-09-21T14:24:45.176+02:00","recording_url":"https://cdn.media.ccc.de/events/all_systems_go/2019/h264-hd/asg2019-159-eng-Traceloop_for_systemd_and_Kubernetes_Inspektor_Gadget_hd.mp4","url":"https://api.media.ccc.de/public/recordings/40310","event_url":"https://api.media.ccc.de/public/events/2dd3d338-37ae-5b15-8292-1f124c0235a4","conference_url":"https://api.media.ccc.de/public/conferences/asg2019"},{"size":26,"length":2250,"mime_type":"audio/opus","language":"eng","filename":"asg2019-159-eng-Traceloop_for_systemd_and_Kubernetes_Inspektor_Gadget_opus.opus","state":"new","folder":"opus","high_quality":false,"width":0,"height":0,"updated_at":"2019-09-21T16:30:09.259+02:00","recording_url":"https://cdn.media.ccc.de/events/all_systems_go/2019/opus/asg2019-159-eng-Traceloop_for_systemd_and_Kubernetes_Inspektor_Gadget_opus.opus","url":"https://api.media.ccc.de/public/recordings/40358","event_url":"https://api.media.ccc.de/public/events/2dd3d338-37ae-5b15-8292-1f124c0235a4","conference_url":"https://api.media.ccc.de/public/conferences/asg2019"},{"size":254,"length":2250,"mime_type":"video/webm","language":"eng","filename":"asg2019-159-eng-Traceloop_for_systemd_and_Kubernetes_Inspektor_Gadget_webm-hd.webm","state":"new","folder":"webm-hd","high_quality":true,"width":1920,"height":1080,"updated_at":"2019-09-21T16:30:50.741+02:00","recording_url":"https://cdn.media.ccc.de/events/all_systems_go/2019/webm-hd/asg2019-159-eng-Traceloop_for_systemd_and_Kubernetes_Inspektor_Gadget_webm-hd.webm","url":"https://api.media.ccc.de/public/recordings/40359","event_url":"https://api.media.ccc.de/public/events/2dd3d338-37ae-5b15-8292-1f124c0235a4","conference_url":"https://api.media.ccc.de/public/conferences/asg2019"},{"size":91,"length":2250,"mime_type":"video/webm","language":"eng","filename":"asg2019-159-eng-Traceloop_for_systemd_and_Kubernetes_Inspektor_Gadget_webm-sd.webm","state":"new","folder":"webm-sd","high_quality":false,"width":720,"height":576,"updated_at":"2019-09-21T16:30:58.549+02:00","recording_url":"https://cdn.media.ccc.de/events/all_systems_go/2019/webm-sd/asg2019-159-eng-Traceloop_for_systemd_and_Kubernetes_Inspektor_Gadget_webm-sd.webm","url":"https://api.media.ccc.de/public/recordings/40360","event_url":"https://api.media.ccc.de/public/events/2dd3d338-37ae-5b15-8292-1f124c0235a4","conference_url":"https://api.media.ccc.de/public/conferences/asg2019"},{"size":34,"length":2250,"mime_type":"audio/mpeg","language":"eng","filename":"asg2019-159-eng-Traceloop_for_systemd_and_Kubernetes_Inspektor_Gadget_mp3.mp3","state":"new","folder":"mp3","high_quality":false,"width":0,"height":0,"updated_at":"2019-09-21T16:31:36.440+02:00","recording_url":"https://cdn.media.ccc.de/events/all_systems_go/2019/mp3/asg2019-159-eng-Traceloop_for_systemd_and_Kubernetes_Inspektor_Gadget_mp3.mp3","url":"https://api.media.ccc.de/public/recordings/40361","event_url":"https://api.media.ccc.de/public/events/2dd3d338-37ae-5b15-8292-1f124c0235a4","conference_url":"https://api.media.ccc.de/public/conferences/asg2019"},{"size":65,"length":2250,"mime_type":"video/mp4","language":"eng","filename":"asg2019-159-eng-Traceloop_for_systemd_and_Kubernetes_Inspektor_Gadget_sd.mp4","state":"new","folder":"h264-sd","high_quality":false,"width":720,"height":576,"updated_at":"2019-09-21T16:31:56.608+02:00","recording_url":"https://cdn.media.ccc.de/events/all_systems_go/2019/h264-sd/asg2019-159-eng-Traceloop_for_systemd_and_Kubernetes_Inspektor_Gadget_sd.mp4","url":"https://api.media.ccc.de/public/recordings/40362","event_url":"https://api.media.ccc.de/public/events/2dd3d338-37ae-5b15-8292-1f124c0235a4","conference_url":"https://api.media.ccc.de/public/conferences/asg2019"}]}