Live on-demand transcoding in go using ffmpeg. Also with NVIDIA GPU hardware acceleration.

Last update: Dec 16, 2022

Comments: 15

Go live HTTP on-demand transcoding

Transcoding is expensive and resource consuming operation on CPU and GPU. For big companies with thousands of customers it is essential, to have a dedicated 24/7 transcoding servers. But we, single sporadic users of transcoding, need to have different approach. Transcoding should be done only when its output is really needed. This tool is trying to solve this problem by offering transcoding on demand.

This tool is intended to be used with live streams only. Seeking is not supported, yet.

Config

Specify streams as object in yaml file.

Streams

Create streams.yaml file, with your streams:

streams:
  :

Example:

streams:
  cam: rtmp://localhost/live/cam
  ch1_hd: http://192.168.1.34:9981/stream/channelid/85
  ch2_hd: http://192.168.1.34:9981/stream/channelid/43

HTTP streaming is accessible via:

http://localhost:8080//

HLS is accessible via:

http://localhost:8080///index.m3u8
http://localhost:8080///play.html

CPU Profiles

Profiles (HTTP and HLS) with CPU transcoding can be found in profiles:

h264_360p
h264_540p
h264_720p
h264_1080p

Profile names must match flowing regex: ^[0-9A-Za-z_-]+$

GPU Profiles

Profiles (HTTP and HLS) with GPU transcoding can be found in profiles_nvidia:

h264_360p
h264_540p
h264_720p
h264_1080p

Profile names must match flowing regex: ^[0-9A-Za-z_-]+$

Docker

Build

docker build -t go-transcode:latest .

Run

docker run --rm -d \
  --name="go-transcode" \
  -p "8080:8080" \
  -v "${PWD}/streams.yaml:/app/streams.yaml" go-transcode:latest

Nvidia GPU support (docker)

You will need to have nvidia-docker installed.

Build

First, you need to build previous container. Then, build Nvidia container.

docker build --build-arg "TRANSCODE_IMAGE=go-transcode:latest" -t go-transcode-nvidia:latest -f Dockerfile.nvidia .

Run

docker run --rm -d \
  --gpus=all \
  --name="go-transcode-nvidia" \
  -p "8080:8080" \
  -v "${PWD}/streams.yaml:/app/streams.yaml" go-transcode-nvidia:latest

Supported inputs

Input codec will be automatically determined from given stream. Please check your graphic card's supported codec and maximum concurrent sessions here.

Codec	CUVID	Codec Name
h264	h264_cuvid	H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
hevc	hevc_cuvid	H.265 / HEVC
mjpeg	mjpeg_cuvid	Motion JPEG
mpeg1video	mpeg1_cuvid	MPEG-1 video
mpeg2video	mpeg2_cuvid	MPEG-2 video
mpeg4	mpeg4_cuvid	MPEG-4 part 2
vc1	vc1_cuvid	SMPTE VC-1
vp8	vp8_cuvid	On2 VP8
vp9	vp9_cuvid	Google VP9

Owner

Miroslav Šedivý

https://github.com/m1k1o/go-transcode

Comments

transcode static video files
inspiration hls-vod-too

ffprobe to find duration and segment count

transcode/cache by segment number in file

edit: for static streams maybe config should support directories? library: /media/video in config can serve all files/directories with /profile/library/folder/video.mkv/index.m3u8 (or other url format from #13)
2nd+ streams lead to errors that can cause some players to not work.

The first person to get the stream works, video plays back fine. any one after will get errors that cause the video to freeze or the player to refuse to play

See the attached video

https://user-images.githubusercontent.com/39884745/141701010-e01b7afc-674c-4485-b930-6abff0671df8.mp4

This happens on lan and over hls proxy. I have tried changing the type of audio as well happens to both aac and mp3 Using the latest git docker build as well.
Some refactoring
[x] streams are loaded from viper config, not streams.yaml

[x] cmd is built by go build in repo root (main package)

[x] support /etc/transcode

[x] remove hardcoded paths

[x] config file is loaded from /etc/transcode or ./

[x] profiles/data is served from /etc/transcode or ./

[x] everything is relative to basedir, which is passed via CLI, or is /etc/transcode (if exists), or is ./

[x] config file is automatically reloaded in serve mode

[x] remove some unused stuff

[x] removed Config type

[x] merge some small files together for clarity

[x] ~~remove Server.Proxy unused~~ (moved to #4)

[x] ~~remove TLS support (Server.Key and Server.Cert) encourage reverse proxy~~ add warning that TLS mode should not be used for security

[x] ~~move CMD parsing in internal/config to cmd package~~ future

[x] fix Docker build

[x] Write some tests

[x] settings cascade (test_args.sh)

[x] settings auto-reload

[x] Use settings for ffmpeg profiles folder

[x] Move all to profiles/default and profiles/nvidia

[x] Use a profiles directory setting to find the profiles

[x] make room in repo root

[x] embed data/play.html in binary from internal/api/hls/

[x] move data/*-test.sh to tests/

[x] remove data dir from repo

[x] remove bin dir from repo

[x] ~~move everything docker to packaging/docker dir?~~ future

[x] ~~transcodeStart takes URL not stream name~~ future

[x] Update readme

[x] build and run with docker

[x] build and run without docker

[x] info about project structure

[x] ~~info about code flow for request process~~ future

feedback welcome (i'm new to golang)

also there's a lot of stuff in root dir it's littlee confusing, do you like to move source to src/ (not sure golang likes that), build/test scripts (dev/ data/ and tests) to bin/, and profiles mixed together, like this:

bin/

src/

profiles/

what about docker files? is there a standard directory to place it?

LICENSE README.md go.mod go.sum ...
Unrecognized option 'hls_wrap' from ffmpeg

Hi there,

Just made a fresh build and getting the below error when trying to do any copy or transcode stream, logging into the container and running the same command gives the below response:

bash-5.2# /app/profiles/hls/copy.sh url=http://pathtostream
Unrecognized option 'hls_wrap'. Error splitting the argument list: Option not found

ffmpeg version: bash-5.2# ffmpeg ffmpeg version 5.1.2 Copyright (c) 2000-2022 the FFmpeg developers
VAAPI support

Updated hls_h264 and http to support vaapi detection Updated readme on how to add dri devices to containers added helper vaapi script to detect supported encoders/decoders updated Dockerfile to use alpine edge to get access to drivers and utilities added alpine edge testing repo to get access to drivers and utilities
Auto-detect hardware acceleration
We can detect hardware acceleration in program. Using ffmpeg cli

ffmpeg -init_hw_device list 2> /dev/null Supported hardware device types: vdpau vaapi qsv drm opencl

then no need for profiles config just print what profiles are used on startup

what outputs the command with nvidia proprietaire driver?

Error on probe

Error: 4:47PM ERR unable to load metadata error="unable probe video for keyframes: strconv.ParseFloat: parsing "": invalid syntax"

When I do the ffprobe in terminal I get this result: ffprobe -v error -skip_frame nokey -show_entries frame=pkt_pts_time,side_data_list -show_entries format=duration -show_entries stream=duration,width,height -select_streams v -of json MYVIDEO.mkv

{
    "frames": [
        {
            "side_data_list": [
                {

                },
                {

                }
            ]
        },
        {
            "pkt_pts_time": "0.000000"
        },
        {
            "pkt_pts_time": "2.002000"
        },
        {
            "pkt_pts_time": "3.712000"
        },
        {
            "pkt_pts_time": "8.592000"
        },
.............................

Could be the side_data_list?

Fix invalid segment times

I'm not familiar with this code, but I needed this change to get anything to work. Basically the limit was larger than the size of the breakpoints slice, so it ended up with a zero at the end for segmentTimes. This cause the ffmpeg call to fail because the end time is zero. If it matters, I'm testing with very short videos (~10s)
Add pprof debug endpoint
This endpoint should help with debugging in the future.

More info here: https://pkg.go.dev/net/http/pprof

Types of profiles available:

allocs

block

cmdline

goroutine

heap

mutex

profile

threadcreate

trace

Profile Descriptions:

allocs:
A sampling of all past memory allocations

block:
Stack traces that led to blocking on synchronization primitives

cmdline:
The command line invocation of the current program

goroutine:
Stack traces of all current goroutines

heap:
A sampling of memory allocations of live objects. You can specify the gc GET parameter to run GC before taking the heap sample.

mutex:
Stack traces of holders of contended mutexes

profile:
CPU profile. You can specify the duration in the seconds GET parameter. After you get the profile file, use the go tool pprof command to investigate the profile.

threadcreate:
Stack traces that led to the creation of new OS threads

trace:
A trace of execution of the current program. You can specify the duration in the seconds GET parameter. After you get the trace file, use the go tool trace command to investigate the trace.
HLS exit on error

Right now, when command finishes earlier (because of error) or does not start properly, it won't communincate error and we need to wait for timeout to finish. I would expect the function onStop to have error parameter, and firing when command stops.

Questions on simultaneous ffmpeg processes (NVENC job concurrency) and segment buffering behavior

Is it possible that an issue exists with overlapping transcoding runs, i.e. is it possible that there is a situation (race condition?) in which segmentBufferMax+1 ffmpeg instances run simultaneously?

I'm using the VOD mode with (modified) GPU encoding via NVENC. Consumer-grade Nvidia GPUs only allow 2 or 3 (apparently they increased this to 3 some time in 2020) simultaneous NVENC jobs. I'm testing with a GTX 1050 and a GTX 1070.

I've set segmentBufferMax to 3, otherwise I would constantly run into this issue. With that, at the beginning it runs just fine and does a couple of transcodes successfully. Eventually, it runs into the telltale 'nvenc concurrency' error:

2022/12/08 14:32:09 [h264_nvenc @ 00000273a548f1c0] OpenEncodeSessionEx failed: out of memory (10): (no details)
2022/12/08 14:32:09 [h264_nvenc @ 00000273a548f1c0] No capable devices found
2022/12/08 14:32:09 Error initializing output stream 0:1 -- Error while opening encoder for output stream #0:1 - maybe incorrect parameters such as bit_rate, rate, width or height
2022/12/08 14:32:09 [h264_nvenc @ 00000282534df1c0] OpenEncodeSessionEx failed: out of memory (10): (no details)
2022/12/08 14:32:09 [h264_nvenc @ 00000282534df1c0] No capable devices found
2022/12/08 14:32:09 Error initializing output stream 0:1 -- Error while opening encoder for output stream #0:1 - maybe incorrect parameters such as bit_rate, rate, width or height

After that, go-transcode runs into some kind of hiccup where from now on until a restart, it only produces timeouts:

1:32PM WRN media transcode timeouted module=hlsvod submodule=manager

I'm not sure why this is, but apparently it doesn't handle the ffmpeg failure all too well.

Btw., I'm a bit confused, but from the code comments, segmentBufferMin has nothing to do with this, correct? segmentBufferMin and segmentBufferMax are two completely different configuration values that don't work together to create some kind of range.

Click to expand full log

1:26PM INF new hls vod request hlsResource=1080p.m3u8 module=hlsvod path="video.mp4/1080p.m3u8" vodMediaPath="../test_videos/h/video.mp4"
1:26PM INF new hls vod request hlsResource=1080p-00000.ts module=hlsvod path="video.mp4/1080p-00000.ts" vodMediaPath="../test_videos/h/video.mp4"
1:26PM INF new hls vod request hlsResource=1080p-00001.ts module=hlsvod path="video.mp4/1080p-00001.ts" vodMediaPath="../test_videos/h/video.mp4"
1:26PM INF new hls vod request hlsResource=1080p.m3u8 module=hlsvod path="video.mp4/1080p.m3u8" vodMediaPath="../test_videos/h/video.mp4"
1:26PM INF new hls vod request hlsResource=1080p-00000.ts module=hlsvod path="video.mp4/1080p-00000.ts" vodMediaPath="../test_videos/h/video.mp4"
1:26PM INF new hls vod request hlsResource=1080p-00002.ts module=hlsvod path="video.mp4/1080p-00002.ts" vodMediaPath="../test_videos/h/video.mp4"
1:26PM INF new hls vod request hlsResource=1080p-00001.ts module=hlsvod path="video.mp4/1080p-00001.ts" vodMediaPath="../test_videos/h/video.mp4"
1:26PM INF new hls vod request hlsResource=1080p-00003.ts module=hlsvod path="video.mp4/1080p-00003.ts" vodMediaPath="../test_videos/h/video.mp4"
1:26PM INF new hls vod request hlsResource=1080p-00002.ts module=hlsvod path="video.mp4/1080p-00002.ts" vodMediaPath="../test_videos/h/video.mp4"
1:26PM INF new hls vod request hlsResource=1080p-00003.ts module=hlsvod path="video.mp4/1080p-00003.ts" vodMediaPath="../test_videos/h/video.mp4"
1:26PM INF new hls vod request hlsResource=1080p-00018.ts module=hlsvod path="video.mp4/1080p-00018.ts" vodMediaPath="../test_videos/h/video.mp4"
1:26PM INF new hls vod request hlsResource=1080p-00019.ts module=hlsvod path="video.mp4/1080p-00019.ts" vodMediaPath="../test_videos/h/video.mp4"
1:26PM INF new hls vod request hlsResource=1080p-00018.ts module=hlsvod path="video.mp4/1080p-00018.ts" vodMediaPath="../test_videos/h/video.mp4"
1:26PM INF new hls vod request hlsResource=1080p-00019.ts module=hlsvod path="video.mp4/1080p-00019.ts" vodMediaPath="../test_videos/h/video.mp4"
1:26PM INF new hls vod request hlsResource=1080p-00020.ts module=hlsvod path="video.mp4/1080p-00020.ts" vodMediaPath="../test_videos/h/video.mp4"
1:26PM INF new hls vod request hlsResource=1080p-00020.ts module=hlsvod path="video.mp4/1080p-00020.ts" vodMediaPath="../test_videos/h/video.mp4"
1:26PM INF new hls vod request hlsResource=1080p-00021.ts module=hlsvod path="video.mp4/1080p-00021.ts" vodMediaPath="../test_videos/h/video.mp4"
1:26PM INF new hls vod request hlsResource=1080p-00021.ts module=hlsvod path="video.mp4/1080p-00021.ts" vodMediaPath="../test_videos/h/video.mp4"
1:26PM INF new hls vod request hlsResource=1080p-00022.ts module=hlsvod path="video.mp4/1080p-00022.ts" vodMediaPath="../test_videos/h/video.mp4"
1:26PM INF new hls vod request hlsResource=1080p-00022.ts module=hlsvod path="video.mp4/1080p-00022.ts" vodMediaPath="../test_videos/h/video.mp4"
1:26PM INF new hls vod request hlsResource=1080p-00023.ts module=hlsvod path="video.mp4/1080p-00023.ts" vodMediaPath="../test_videos/h/video.mp4"
1:26PM INF new hls vod request hlsResource=1080p-00023.ts module=hlsvod path="video.mp4/1080p-00023.ts" vodMediaPath="../test_videos/h/video.mp4"
1:31PM INF new hls vod request hlsResource=1080p-00012.ts module=hlsvod path="video.mp4/1080p-00012.ts" vodMediaPath="../test_videos/h/video.mp4"
1:31PM INF transcoding segments limit=1 module=hlsvod offset=12 segments-times=[96,104] submodule=manager
2022/12/08 14:31:55 Starting FFmpeg process with args ./ffmpeg.exe -loglevel warning -ss 96.000000 -i ../test_videos/h/video.mp4 -to 104.000000 -copyts -force_key_frames 104.000000 -sn -vf scale=-2:1080 -filter_complex amix=inputs=2 -c:v h264_nvenc -preset p1 -tune:v ull -profile:v high -rc:v cbr -b:v 5000k -c:a aac -b:a 192k -f segment -segment_time_delta 0.2 -segment_format mpegts -segment_times 104.000000 -segment_start_number 12 -segment_list_type flat -segment_list pipe:1 ../test_videos/h/transcode\vod-1080p-47964795/1080p-%05d.ts
1:31PM INF transcode process started limit=1 module=hlsvod offset=12 submodule=manager
1:31PM INF new hls vod request hlsResource=1080p-00012.ts module=hlsvod path="video.mp4/1080p-00012.ts" vodMediaPath="../test_videos/h/video.mp4"
1:31PM INF transcode process returned a segment index=12 limit=1 module=hlsvod offset=12 segment=1080p-00012.ts submodule=manager
2022/12/08 14:31:59 FFmpeg process successfully finished.
1:31PM INF new hls vod request hlsResource=1080p-00013.ts module=hlsvod path="video.mp4/1080p-00013.ts" vodMediaPath="../test_videos/h/video.mp4"
1:31PM INF transcode process finished index=13 limit=1 module=hlsvod offset=12 submodule=manager
1:31PM INF transcoding segments limit=1 module=hlsvod offset=13 segments-times=[104,112] submodule=manager
2022/12/08 14:31:59 Starting FFmpeg process with args ./ffmpeg.exe -loglevel warning -ss 104.000000 -i ../test_videos/h/video.mp4 -to 112.000000 -copyts -force_key_frames 112.000000 -sn -vf scale=-2:1080 -filter_complex amix=inputs=2 -c:v h264_nvenc -preset p1 -tune:v ull -profile:v high -rc:v cbr -b:v 5000k -c:a aac -b:a 192k -f segment -segment_time_delta 0.2 -segment_format mpegts -segment_times 112.000000 -segment_start_number 13 -segment_list_type flat -segment_list pipe:1 ../test_videos/h/transcode\vod-1080p-47964795/1080p-%05d.ts
1:31PM INF transcode process started limit=1 module=hlsvod offset=13 submodule=manager
1:31PM INF new hls vod request hlsResource=1080p-00013.ts module=hlsvod path="video.mp4/1080p-00013.ts" vodMediaPath="../test_videos/h/video.mp4"
1:32PM INF transcode process returned a segment index=13 limit=1 module=hlsvod offset=13 segment=1080p-00013.ts submodule=manager
2022/12/08 14:32:04 FFmpeg process successfully finished.
1:32PM INF new hls vod request hlsResource=1080p-00014.ts module=hlsvod path="video.mp4/1080p-00014.ts" vodMediaPath="../test_videos/h/video.mp4"
1:32PM INF transcoding segments limit=1 module=hlsvod offset=14 segments-times=[112,120] submodule=manager
1:32PM INF transcode process finished index=14 limit=1 module=hlsvod offset=13 submodule=manager
2022/12/08 14:32:04 Starting FFmpeg process with args ./ffmpeg.exe -loglevel warning -ss 112.000000 -i ../test_videos/h/video.mp4 -to 120.000000 -copyts -force_key_frames 120.000000 -sn -vf scale=-2:1080 -filter_complex amix=inputs=2 -c:v h264_nvenc -preset p1 -tune:v ull -profile:v high -rc:v cbr -b:v 5000k -c:a aac -b:a 192k -f segment -segment_time_delta 0.2 -segment_format mpegts -segment_times 120.000000 -segment_start_number 14 -segment_list_type flat -segment_list pipe:1 ../test_videos/h/transcode\vod-1080p-47964795/1080p-%05d.ts
1:32PM INF transcode process started limit=1 module=hlsvod offset=14 submodule=manager
1:32PM INF new hls vod request hlsResource=1080p-00014.ts module=hlsvod path="video.mp4/1080p-00014.ts" vodMediaPath="../test_videos/h/video.mp4"
1:32PM INF new hls vod request hlsResource=1080p-00015.ts module=hlsvod path="video.mp4/1080p-00015.ts" vodMediaPath="../test_videos/h/video.mp4"
1:32PM INF transcoding segments limit=1 module=hlsvod offset=15 segments-times=[120,128] submodule=manager
2022/12/08 14:32:04 Starting FFmpeg process with args ./ffmpeg.exe -loglevel warning -ss 120.000000 -i ../test_videos/h/video.mp4 -to 128.000000 -copyts -force_key_frames 128.000000 -sn -vf scale=-2:1080 -filter_complex amix=inputs=2 -c:v h264_nvenc -preset p1 -tune:v ull -profile:v high -rc:v cbr -b:v 5000k -c:a aac -b:a 192k -f segment -segment_time_delta 0.2 -segment_format mpegts -segment_times 128.000000 -segment_start_number 15 -segment_list_type flat -segment_list pipe:1 ../test_videos/h/transcode\vod-1080p-47964795/1080p-%05d.ts
1:32PM INF new hls vod request hlsResource=1080p-00015.ts module=hlsvod path="video.mp4/1080p-00015.ts" vodMediaPath="../test_videos/h/video.mp4"
1:32PM INF transcoding segments limit=1 module=hlsvod offset=15 segments-times=[120,128] submodule=manager
2022/12/08 14:32:04 Starting FFmpeg process with args ./ffmpeg.exe -loglevel warning -ss 120.000000 -i ../test_videos/h/video.mp4 -to 128.000000 -copyts -force_key_frames 128.000000 -sn -vf scale=-2:1080 -filter_complex amix=inputs=2 -c:v h264_nvenc -preset p1 -tune:v ull -profile:v high -rc:v cbr -b:v 5000k -c:a aac -b:a 192k -f segment -segment_time_delta 0.2 -segment_format mpegts -segment_times 128.000000 -segment_start_number 15 -segment_list_type flat -segment_list pipe:1 ../test_videos/h/transcode\vod-1080p-47964795/1080p-%05d.ts
1:32PM INF transcode process started limit=1 module=hlsvod offset=15 submodule=manager
1:32PM INF transcode process started limit=1 module=hlsvod offset=15 submodule=manager
1:32PM INF new hls vod request hlsResource=1080p-00016.ts module=hlsvod path="video.mp4/1080p-00016.ts" vodMediaPath="../test_videos/h/video.mp4"
1:32PM INF transcoding segments limit=1 module=hlsvod offset=16 segments-times=[128,136] submodule=manager
2022/12/08 14:32:08 Starting FFmpeg process with args ./ffmpeg.exe -loglevel warning -ss 128.000000 -i ../test_videos/h/video.mp4 -to 136.000000 -copyts -force_key_frames 136.000000 -sn -vf scale=-2:1080 -filter_complex amix=inputs=2 -c:v h264_nvenc -preset p1 -tune:v ull -profile:v high -rc:v cbr -b:v 5000k -c:a aac -b:a 192k -f segment -segment_time_delta 0.2 -segment_format mpegts -segment_times 136.000000 -segment_start_number 16 -segment_list_type flat -segment_list pipe:1 ../test_videos/h/transcode\vod-1080p-47964795/1080p-%05d.ts
1:32PM INF new hls vod request hlsResource=1080p-00016.ts module=hlsvod path="video.mp4/1080p-00016.ts" vodMediaPath="../test_videos/h/video.mp4"
1:32PM INF transcoding segments limit=1 module=hlsvod offset=16 segments-times=[128,136] submodule=manager
2022/12/08 14:32:08 Starting FFmpeg process with args ./ffmpeg.exe -loglevel warning -ss 128.000000 -i ../test_videos/h/video.mp4 -to 136.000000 -copyts -force_key_frames 136.000000 -sn -vf scale=-2:1080 -filter_complex amix=inputs=2 -c:v h264_nvenc -preset p1 -tune:v ull -profile:v high -rc:v cbr -b:v 5000k -c:a aac -b:a 192k -f segment -segment_time_delta 0.2 -segment_format mpegts -segment_times 136.000000 -segment_start_number 16 -segment_list_type flat -segment_list pipe:1 ../test_videos/h/transcode\vod-1080p-47964795/1080p-%05d.ts
1:32PM INF transcode process started limit=1 module=hlsvod offset=16 submodule=manager
1:32PM INF transcode process started limit=1 module=hlsvod offset=16 submodule=manager
2022/12/08 14:32:09 [h264_nvenc @ 00000273a548f1c0] OpenEncodeSessionEx failed: out of memory (10): (no details)
2022/12/08 14:32:09 [h264_nvenc @ 00000273a548f1c0] No capable devices found
2022/12/08 14:32:09 Error initializing output stream 0:1 -- Error while opening encoder for output stream #0:1 - maybe incorrect parameters such as bit_rate, rate, width or height
2022/12/08 14:32:09 [h264_nvenc @ 00000282534df1c0] OpenEncodeSessionEx failed: out of memory (10): (no details)
2022/12/08 14:32:09 [h264_nvenc @ 00000282534df1c0] No capable devices found
2022/12/08 14:32:09 Error initializing output stream 0:1 -- Error while opening encoder for output stream #0:1 - maybe incorrect parameters such as bit_rate, rate, width or height
2022/12/08 14:32:09 FFmpeg process exited with error: exit status 1
1:32PM INF transcode process finished index=16 limit=1 module=hlsvod offset=16 submodule=manager
2022/12/08 14:32:09 FFmpeg process exited with error: exit status 1
1:32PM INF transcode process finished index=16 limit=1 module=hlsvod offset=16 submodule=manager
1:32PM INF transcode process returned a segment index=14 limit=1 module=hlsvod offset=14 segment=1080p-00014.ts submodule=manager
2022/12/08 14:32:10 FFmpeg process successfully finished.
1:32PM INF transcode process finished index=15 limit=1 module=hlsvod offset=14 submodule=manager
1:32PM INF transcode process returned a segment index=15 limit=1 module=hlsvod offset=15 segment=1080p-00015.ts submodule=manager
1:32PM INF transcode process returned a segment index=15 limit=1 module=hlsvod offset=15 segment=1080p-00015.ts submodule=manager
2022/12/08 14:32:11 FFmpeg process successfully finished.
1:32PM INF transcode process finished index=16 limit=1 module=hlsvod offset=15 submodule=manager
2022/12/08 14:32:11 FFmpeg process successfully finished.
1:32PM INF transcode process finished index=16 limit=1 module=hlsvod offset=15 submodule=manager
1:32PM WRN media transcode timeouted module=hlsvod submodule=manager
1:32PM WRN media transcode timeouted module=hlsvod submodule=manager
1:32PM WRN media transcode timeouted module=hlsvod submodule=manager
1:32PM INF new hls vod request hlsResource=1080p-00016.ts module=hlsvod path="video.mp4/1080p-00016.ts" vodMediaPath="../test_videos/h/video.mp4"
1:32PM INF new hls vod request hlsResource=1080p-00016.ts module=hlsvod path="video.mp4/1080p-00016.ts" vodMediaPath="../test_videos/h/video.mp4"
1:32PM WRN media transcode timeouted module=hlsvod submodule=manager
1:32PM WRN media transcode timeouted module=hlsvod submodule=manager
1:32PM INF new hls vod request hlsResource=1080p-00016.ts module=hlsvod path="video.mp4/1080p-00016.ts" vodMediaPath="../test_videos/h/video.mp4"
1:32PM INF new hls vod request hlsResource=1080p-00016.ts module=hlsvod path="video.mp4/1080p-00016.ts" vodMediaPath="../test_videos/h/video.mp4"
1:32PM WRN media transcode timeouted module=hlsvod submodule=manager
1:32PM WRN media transcode timeouted module=hlsvod submodule=manager
1:32PM INF new hls vod request hlsResource=1080p-00016.ts module=hlsvod path="video.mp4/1080p-00016.ts" vodMediaPath="../test_videos/h/video.mp4"
1:32PM INF new hls vod request hlsResource=1080p-00016.ts module=hlsvod path="video.mp4/1080p-00016.ts" vodMediaPath="../test_videos/h/video.mp4"
1:32PM WRN media transcode timeouted module=hlsvod submodule=manager
1:32PM WRN media transcode timeouted module=hlsvod submodule=manager
1:32PM INF new hls vod request hlsResource=1080p-00016.ts module=hlsvod path="video.mp4/1080p-00016.ts" vodMediaPath="../test_videos/h/video.mp4"
1:32PM INF new hls vod request hlsResource=1080p-00016.ts module=hlsvod path="video.mp4/1080p-00016.ts" vodMediaPath="../test_videos/h/video.mp4"
1:32PM WRN media transcode timeouted module=hlsvod submodule=manager
1:32PM WRN media transcode timeouted module=hlsvod submodule=manager
1:32PM INF new hls vod request hlsResource=1080p-00016.ts module=hlsvod path="video.mp4/1080p-00016.ts" vodMediaPath="../test_videos/h/video.mp4"
1:32PM INF new hls vod request hlsResource=1080p-00016.ts module=hlsvod path="video.mp4/1080p-00016.ts" vodMediaPath="../test_videos/h/video.mp4"
1:33PM WRN media transcode timeouted module=hlsvod submodule=manager
1:33PM WRN media transcode timeouted module=hlsvod submodule=manager
1:33PM INF new hls vod request hlsResource=1080p-00016.ts module=hlsvod path="video.mp4/1080p-00016.ts" vodMediaPath="../test_videos/h/video.mp4"
1:33PM INF new hls vod request hlsResource=1080p-00016.ts module=hlsvod path="video.mp4/1080p-00016.ts" vodMediaPath="../test_videos/h/video.mp4"
1:33PM WRN media transcode timeouted module=hlsvod submodule=manager
1:33PM WRN media transcode timeouted module=hlsvod submodule=manager

Add support for running go-transcode on Windows

I know this probably isn't very high on your list, but I have a use case for on-demand transcoding on Windows, couldn't find any other existing project doing this as easily as this one, and it works with some minor changes, so… here we go!

Let me know what you think and how you want to proceed with this. I see this primarily as a test balloon, to figure out whether changes like this are something you're interested in merging upstream? I could understand if this isn't something you want to deal with (and I'm probably not interested in maintaining this long-term either). I guess I'd just use my fork for my upcoming project and not open further pull requests related to Windows-support.

Some technical notes:

Really the only platform-dependent bits of code were the process group management. I've replaced those with Windows-specific versions, although I haven't fully tested them, because I'm not entirely sure how they're used. (I've pressed Ctrl-C on a running instance once and it seemed to terminate the running ffmpeg instances. But then again, I think they were normal individual processes, so I don't see the need for project groups yet.) Also, the build configuration for Linux and other non-Windows platforms is completely untested.

I believe I also got stream support working, by mashing together some of the bash scripts into a Windows Batch file and replacing that one bit where it concats .sh to the profile name with .cmd. However, I'm only doing VOD and I just found out, that I apparently don't even need those bash scripts, so it isn't included here.

Note that this is the first time I've written or built any Go, so my code is probably not very idiomatic. Suggestions (including required reading) welcome.
Add hardware acceleration support for static files

I'm using --device=/dev/dri:/dev/dri in my compose file but it doesn't seem to make any difference in CPU usage vs leaving it out. Am I missing some other config? Thanks

Ffmpeg does not force key frames as expected

Given we have a test video and want to force keframes at 1772,1776 timestamps:

ffmpeg -loglevel warning \
    -ss 1768 \
    -i test.mp4 \
    -force_key_frames 1772,1776 \
    -to 1776 \
    -copyts \
  -c:v libx264 \
    -preset faster \
    -profile:v high \
    -level:v 4.0 \
    -b:v 2800k \
  -c:a aac \
    -b:a 192k \
  -f mpegts - \
| ffprobe -loglevel error -skip_frame nokey -select_streams v:0 -show_entries frame=pkt_pts_time -of csv=print_section=0 -

Response shows us:

1769.416256
1773.420256
1777.382544

What is not what we expected. We expected to see 1772 and 1776 in the output.

static file access

Trying to test static files. the config file is pointed to /media has full access to the folder mounted

mpv http://IPADDR:8888/vod/TV/SouthPark/Specials/South_Park_-_S00E44_-_South_Park-_Post_COVID-_The_Return_of_COVID_WEBDL-1080p.mkv/index.m3u8

get an error 500 and the output with debug on on go-transcode

12:46AM INF fetching metadata module=hlsvod submodule=manager
2021/12/28 00:46:13 Unrecognized option 'show_format'.
Error splitting the argument list: Option not found

12:46AM WRN unable to preload metadata error="unable probe media for metadata: exit status 1" module=hlsvod
12:46AM DBG request complete (500) module=http req={"agent":"libmpv","id":"go-transcode/NeOacMNoCW-000004","method":"GET","proto":"HTTP/1.1","remote":"192.168.6.133:1082","scheme":"http","uri":"http://IPADDR:8888/vod/TV/SouthPark/Specials/South_Park_-_S00E44_-_South_Park-_Post_COVID-_The_Return_of_COVID_WEBDL-1080p.mkv/index.m3u8"} res={"bytes":31,"elapsed":11.91186,"status":500,"time":"Tue, 28 Dec 2021 00:46:13 UTC"}
12:46AM DBG request complete (405) module=http req={"agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36","id":"go-transcode/NeOacMNoCW-000005","method":"HEAD","proto":"HTTP/1.1","remote":"192.168.6.133:1083","scheme":"http","uri":"http://IPADDR:8888/vod/TV/SouthPark/Specials/South_Park_-_S00E44_-_South_Park-_Post_COVID-_The_Return_of_COVID_WEBDL-1080p.mkv/index.m3u8"} res={"bytes":0,"elapsed":0.013375,"status":405,"time":"Tue, 28 Dec 2021 00:46:14 UTC"}
12:46AM INF fetching metadata module=hlsvod submodule=manager
2021/12/28 00:46:14 Unrecognized option 'show_format'.
Error splitting the argument list: Option not found

12:46AM WRN unable to preload metadata error="unable probe media for metadata: exit status 1" module=hlsvod
12:46AM DBG request complete (500) module=http req={"agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36","id":"go-transcode/NeOacMNoCW-000006","method":"GET","proto":"HTTP/1.1","remote":"192.168.6.133:1084","scheme":"http","uri":"http://IPADDR:8888/vod/TV/SouthPark/Specials/South_Park_-_S00E44_-_South_Park-_Post_COVID-_The_Return_of_COVID_WEBDL-1080p.mkv/index.m3u8"} res={"bytes":31,"elapsed":11.783193,"status":500,"time":"Tue, 28 Dec 2021 00:46:14 UTC"}

[WIP] a very simple, tiny and intuitive ffmpeg wrapper with a cli interface for inspecting & transforming media files supported by the original ffmpeg software

About a very simple, tiny and intuitive ffmpeg wrapper with a cli interface for inspecting & transforming media files supported by the original ffmpeg

Oct 21, 2022

A small program in Go that efficiently compresses videos using ffmpeg.

discordcompressor A small program in Go that efficiently compresses videos using ffmpeg. Dependencies FFmpeg including FFprobe Usage discordcompressor

Dec 18, 2022

golang function that download a video from youtube, and convert it to a mp3 file using ffmpeg

echedwnmp3 echedwnmp3 is a function that download a video from youtube, and convert it to a mp3 file using ffmpeg example package main import(echedwn

Dec 7, 2021

lmmp3 is a little golang library that download a video from youtube, and convert it to a mp3 file using ffmpeg

lmmp3 lmmp3 is a function that download a video from youtube, and convert it to a mp3 file using ffmpeg You need to have installed ffmpeg in your syst

Aug 12, 2022

Golang bindings for FFmpeg

goav Golang binding for FFmpeg A comprehensive binding to the ffmpeg video/audio manipulation library. Usage import "github.com/giorgisio/goav/avforma

Dec 27, 2022

A Go implementation of fluent-ffmpeg

Dec 7, 2022

Stream video from ffmpeg to webrtc

ffmpeg-to-webrtc demonstrates how to send video from ffmpeg to your browser using pion.

Dec 28, 2022

Videncode - Media Encoder (with ffmpeg)

Videncode - Media Encoder (with ffmpeg) Powered by yellyoshua (With2 easy steps) - Build JSON with folder of videos > Process the videos to the new fo

Nov 19, 2022

ffmpeg core for golang

go-ffmpeg-core ffmpeg core for golang 基于ffmpeg命令封装简单功能，因此运行环境需事先安装有ffmpeg命令。 ffmpeg官方下载地址: http://ffmpeg.org/download.html 1. 剥离视频文件的音频/视频 package mai

Nov 26, 2021

ffcommander - An easy frontend to FFmpeg and Imagemagick to automatically process video and manipulate subtitles.

May 9, 2022

A go program that relies on back-end ffmpeg to process video-related content

Video Compress A go program that relies on back-end ffmpeg to process video-related content Installation v-go You can download the corresponding v-go

Dec 22, 2021

Parse and generate m3u8 playlists for Apple HTTP Live Streaming (HLS) in Golang (ported from gem https://github.com/sethdeckard/m3u8)

go-m3u8 Golang package for m3u8 (ported m3u8 gem https://github.com/sethdeckard/m3u8) go-m3u8 provides easy generation and parsing of m3u8 playlists d

Nov 19, 2022

🔥 Golang live stream lib/client/server. support RTMP/RTSP/HLS/HTTP[S]-FLV/HTTP-TS, H264/H265/AAC, relay, cluster, record, HTTP API/Notify, GOP cache. 官方文档见 https://pengrl.com/lal

lal是一个开源GoLang直播流媒体网络传输项目，包含三个主要组成部分： lalserver：流媒体转发服务器。类似于nginx-rtmp-module等应用，但支持更多的协议，提供更丰富的功能。lalserver简介 demo：一些小应用，比如推、拉流客户端，压测工具，流分析工具，调度示例程序等

Jan 1, 2023

Live on-demand transcoding in go using ffmpeg. Also with NVIDIA GPU hardware acceleration.

Go live HTTP on-demand transcoding

Config

Streams

CPU Profiles

GPU Profiles

Docker

Build

Run

Nvidia GPU support (docker)

Build

Run

Supported inputs

Owner

Miroslav Šedivý

Comments

transcode static video files

2nd+ streams lead to errors that can cause some players to not work.

Some refactoring

Unrecognized option 'hls_wrap' from ffmpeg

VAAPI support

Auto-detect hardware acceleration

Error on probe

Fix invalid segment times

Add pprof debug endpoint

HLS exit on error

Questions on simultaneous ffmpeg processes (NVENC job concurrency) and segment buffering behavior

Add support for running go-transcode on Windows

Add hardware acceleration support for static files

Ffmpeg does not force key frames as expected

static file access

Related tags

[WIP] a very simple, tiny and intuitive ffmpeg wrapper with a cli interface for inspecting & transforming media files supported by the original ffmpeg software

A small program in Go that efficiently compresses videos using ffmpeg.

golang function that download a video from youtube, and convert it to a mp3 file using ffmpeg

lmmp3 is a little golang library that download a video from youtube, and convert it to a mp3 file using ffmpeg

Golang bindings for FFmpeg

A Go implementation of fluent-ffmpeg

Stream video from ffmpeg to webrtc

Videncode - Media Encoder (with ffmpeg)

ffmpeg core for golang

ffcommander - An easy frontend to FFmpeg and Imagemagick to automatically process video and manipulate subtitles.

A go program that relies on back-end ffmpeg to process video-related content

Parse and generate m3u8 playlists for Apple HTTP Live Streaming (HLS) in Golang (ported from gem https://github.com/sethdeckard/m3u8)

🔥 Golang live stream lib/client/server. support RTMP/RTSP/HLS/HTTP[S]-FLV/HTTP-TS, H264/H265/AAC, relay, cluster, record, HTTP API/Notify, GOP cache. 官方文档见 https://pengrl.com/lal

Take control over your live stream video by running it yourself. Streaming + chat out of the box.

live video streaming server in golang

live streaming server in golang

A live streaming tool more suitable for non-graphical servers

Plays videos using Prometheus and Grafana, e.g. Bad Apple.

Synthetic media is a realistic transformation of audio and video using artificial intelligence.