Because you liked my first suggestion, I took the liberty of suggesting some more changes to simplify readability. Hope they are welcome and I'd love to hear what you think of them. I also hope I didn't break anything :100:

Problem Statement

From my viewpoint, the biggest thing stopping me from using github.com/pkg/profile is its hard stance on "no more than one kind of profiling at a time". Especially since my usual starting assumption is "show me both a CPU profile and a memory (allocs) profile".

Taking a CPU + memory profile, or a CPU + memory + goroutines + mutexes profile is actually quite a sane thing to do in my viewpoint. Further I do sometimes want to get both a trace and a cpu profile from the same long-running program, rather than go to all the trouble of running it twice.

So with that introduction, let me lay out my understanding of why this is an okay thing to do (conversely, why the restriction on "one at a time" is actually necessary and overly restrictive).

Kinds of Profiling

For the sake of my argument I'm going to focus primarily on the distinction between active and passive profiles.

There are two kinds of active profiles: CPU profiling and execution tracing. Here the cost of instrumentation is so high that they only run for a set amount of time (say take a 10-second CPU profile from a server, or re-running a tool with trace profiling turned on for the duration).

On the other hand there are passive profiles including allocs, block, goroutine, heap, mutex, and threadcreate. Here the instrumentation cost is low enough that these profiles are latently always on; their counts are collected all the time, and can be collected at any point in time.

NOTE this model is of course complicated by the fact that there is a tuning knob for the passive/always-on memory profiler, which pkg/profile is fairly unique in making such a user-facing API-out of (to say nothing of the basically unusable CPU profiler hz tuning knob...)

Common Combinations

Of all those lovely modes of profiling CPU, memory, and tracing are the most generally known, used, and useful; as evidenced by the prevalent flag idioms -cpuprofile FILE, -memprofile FILE, and -trace FILE.

There's really no problem in combining CPU profiling and memory profiling (at normal rates); to the contrary: since the memory profiler is always on any how, you may as well dump it at the end (in the context of an offline tool, which seems to be the use-case that pkg/profile is most suited for). The same goes for any/all of the other passive profilers: they already have their counts just sitting there, you're only doing yourself a harm by not dumping the data.

The most concern comes when combining CPU profiling and tracing. But even there, at least in my experience, any cross-chatter is fairly easy to pick out or compensate for:

• in the event trace, you can at least see that a CPU profile was going on, since there will be at least one dedicated goroutine writing it out over some time span (you should even be able to see the os-interrupt timing... but I've not actually tried to do that, and now I'm purely speculating in parenthetical...)
• there is real concern when it comes to skewing the CPU profile however, since every trace instrumentation branch is now hot, further inflating (the probably already dominant) impact of runtime functions
• in practice, for online servers, I always sequence tracing and CPU profiling for this reason, rather than do them in parallel; however for an offline tool where your profiling its entire lifecycle, there are times when you want to see both concurrently (even if you also re-run it to also get a "pure" trace and cpu profile)

Why I Care

In my view pkg/profile is very close to fully solving the use case of "lifetime profiling for an offline tool / batch task". I'd like to wrap some flag.Value implementation around it and use it as a dependency (maybe even send a pull request for adding the flag convenience).

However in its current form, not being able to take several at once is a bit of a blocker for that use case.

Hi, I'm trying to profile my application using your package following the documentation. I have a cpu.pprof non empty file generated but pprof is unable to read it

$ go tool pprof —pdf ./app /var/folders/x7/wnp2zjn563j2dr45dsfpvdz80000gn/T/profile464041113/cpu.pprof > out.pdf              nsitbon@mac-ns
parsing profile: unrecognized profile format

I've tried multiple configuration from the default one defer profile.Start().Stop() to a more elaborated defer profile.Start(profile.BlockProfile, profile.MemProfile, profile.CPUProfile).Stop() but pprof is still unable to parse it. Any ideas?

FYI

$ go version                                                                                                                  
go version go1.8.3 darwin/amd64

Thanks

I'm trying to profile a simple application:

package main

import (
    "github.com/pkg/profile"
    "github.com/spf13/cobra"
    "strings"
)

/*
Tokenize TODO
*/
func Tokenize(text string) []string {
    return strings.Split(text, "\n")
}

func main() {
    profileCommand := &cobra.Command{
        Use: "profile",
        Run: func(cmd *cobra.Command, args []string) {
            defer profile.Start(profile.CPUProfile, profile.ProfilePath(".")).Stop()

            Tokenize("Hello\nWorld")
        },
    }

    rootCommand := &cobra.Command{Use: "bern"}
    rootCommand.AddCommand(profileCommand)
    rootCommand.Execute()
}

But when I look at the cpu.pprof file it simply contains ' and nothing else. What am I doing incorrectly?

After profiling I get a 10 byte file (cpu.pprof.zip) that when read produces:

• cpu.pprof: decompressing profile: unexpected EOF ( With go tool pprof cpu.pprof)
• Unexpected end of ZLIB input stream (With Goland)

Tried both:

defer profile.Start(profile.ProfilePath(os.Getenv("HOME"))).Stop()

as well as:

p := profile.Start(profile.ProfilePath(os.Getenv("HOME")), profile.NoShutdownHook)
// ...
p.Stop()

As of 3704c8d23324a3fc0771ad2c1cef0aa4e9459f6f on macOS Big Sur 11.1.

With the shutdown hook I saw the library was intercepting shutdown to stop profiling. Without it I see my code called to .Stop().

Troubleshooting ideas?

Thanks for your work.

Now that we've made it impossible to activate more than one profile at a time, users may attempt to work around this with something like

 defer profile.Start(profile.CPUProfile).Stop
 defer profile.Start(profile.MemProfile).Stop

We should decide to support this, and if so, make sure that we move anything which is shared across profiles into the *Profile, see #5. Or, we should prohibit it with some package global lock.

I was looking over this with the hope to learn something new and I found it was a great example of using the functional options you spoke about during your talk!

I thought I'd make this change because I personally find it more readable to use a switch when there are multiple if's, it kind of prepares the reader for a lot of conditions. The fact that Go also allows us to have a "no parameter" switch gives us the opportunity to do such things.

This is obviously just a personal opinion and you don't have to merge it if you don't agree.

profile.Start(profile.MemProfile, profile.ProfilePath("."), profile.NoShutdownHook)
profile.Start(profile.CPUProfile, profile.ProfilePath("."), profile.NoShutdownHook)

Starting multiple profiles throws an error : profile: Start() already called

I'm trying to create a Debian package for profile. The test TestProfile fails during the build of the packages:

go test -v -p 4 github.com/pkg/profile
=== RUN   TestProfile
--- FAIL: TestProfile (1.75s)
profile_test.go:174: default profile (cpu)
profile_test.go:174: memory profile
profile_test.go:174: memory profile (rate 2048)
profile_test.go:174: double start
profile_test.go:174: block profile
profile_test.go:174: profile path
profile_test.go:174: profile path error
profile_test.go:194: stderr: wanted '[could not create initial output]', got '2017/05/09 06:50:47 profile: cpu profiling enabled, README.md/cpu.pprof
	2017/05/09 06:50:47 profile: cpu profiling disabled, README.md/cpu.pprof
	'
profile_test.go:209: expected error
profile_test.go:174: multiple profile sessions
profile_test.go:174: profile quiet

I have been trying to collect CPU and Memory profiles for a program I wrote. I have imported "github.com/pkg/profile" and added the appropriate line for either CPU or Memory profile.

import (
    ...
    "github.com/pkg/profile"
    ...
)

...

func main() {
    // memory profile
    defer profile.Start(profile.MemProfile, profile.ProfilePath("./profiles/")).Stop()
    // OR cpu profile
    defer profile.Start(profile.CPUProfile, profile.ProfilePath("./profiles/")).Stop()
}

When I run the program, I see messages like the following on the command line:

2019/10/11 12:33:24 profile: memory profiling enabled (rate 4096), profiles/mem.pprof

However, the cpu.pprof or mem.pprof files are always 0 bytes. This used to work before for the same program, I do not understand why this no longer works.

ls -alhrt profiles
total 8.0K
-rw-rw-r-- 1 manas manas    0 Oct 11 12:33 mem.pprof
drwxrwxr-x 9 manas manas 4.0K Oct 11 12:33 ..
drwxrwxr-x 2 manas manas 4.0K Oct 11 12:33 .

Fantastic tool.

One thing, say I have a binary that is loading slowly from a massive amount of imports that may possibly be having side effects upon code being loaded.

I don't see how to use profile here because I put defer profile.Start().Stop() at the beginning of main but that doesn't capture the loading/code execution of library code.

thank you

Travis-ci.org is ceased. I just did the migration to GH actions for two of my repos so when I saw this I thought if you also need to move this can help and took me just few minutes.

Ignore if you have other plans.

Failed to remove the files created by the profile pkg "error":"remove mem.pprof: The process cannot access the file because it is being used by another process."

Problem

Currently the signature of the Start function is

func Start(options ...func(*Profile)) interface {
	Stop()
} 

Use case

The lifecycle in our case is managed by another library (fluent-bit-go)

• The Start() has to happen in lifecycle function FLBPluginInit
• The Stop() has to happen in lifecycle function FLBPluginExit

Proposal

have a named interface for Stop

type Stopper interface {
	Stop()
} 

So the Start will become

func Start(options ...func(*Profile)) Stopper

Hi. I hope everyone is having a great new year so far.

I was told by a colleague I could use this package for determining how much time is being spent in each function during an application's run. Is this correct and, if so, how? I saw nothing in the documentation which makes this clear if so.

Thanks in advance.

Hi Team, Added power support for the travis.yml file with ppc64le. This is part of the Ubuntu distribution for ppc64le. This helps us simplify testing later when distributions are re-building and re-releasing. For more info tag @gerrith3. continuous integration build has passed after adding power support arch ppc64le without any failures. please check and close this component.

Every time I use this package, I think that I can do something like:

var flagCPUProfile = flag.String("cpuprofile", "", "write a cpuprofile to `cpuprofile`")

func main() {
	if *flagCPUProfile != "" {
		defer profile.Start(profile.ProfilePath(*flagCPUProfile)).Stop()
	}

And then I run the program with -cpuprofile=somename.pprof. And there is no profile to be found.

And then I dig through the docs and discover ProfilePath is supposed to be a directory. And then I wish there was a way to provide a filename. This is because I sometimes do multiple runs, and I want to write the results to different profiles so that I can combine them, and I don't want to have to deal with creating and cleaning up a directory per profile.

May I send a PR to add ProfileFilename, or something like it?