Hi,

I saw a high memory consumption in our go application that uses the minio admin sdk. I tracked it down to the ListUsers function. It seems the results are not properly garbage collected with multiple calls.

Here is a code snippet and the memory consumption:

ctx := context.Background()
for i := 1; i <= 10; i++ {
    PrintMemUsage()
    _, err := r.MinioAdminClient.ListUsers(ctx)
    if err != nil {
	return nil, err
    }
}

Memory usages: Alloc = 99 MiB TotalAlloc = 123 MiB Sys = 139 MiB NumGC = 9 Alloc = 92 MiB TotalAlloc = 189 MiB Sys = 206 MiB NumGC = 10 Alloc = 156 MiB TotalAlloc = 253 MiB Sys = 206 MiB NumGC = 10 Alloc = 156 MiB TotalAlloc = 253 MiB Sys = 206 MiB NumGC = 10 Alloc = 156 MiB TotalAlloc = 253 MiB Sys = 206 MiB NumGC = 10 Alloc = 220 MiB TotalAlloc = 317 MiB Sys = 272 MiB NumGC = 11 Alloc = 220 MiB TotalAlloc = 318 MiB Sys = 272 MiB NumGC = 11 Alloc = 284 MiB TotalAlloc = 382 MiB Sys = 338 MiB NumGC = 11 Alloc = 284 MiB TotalAlloc = 382 MiB Sys = 338 MiB NumGC = 11 Alloc = 284 MiB TotalAlloc = 382 MiB Sys = 338 MiB NumGC = 11

Can this be investigated?

I am working on packaging this library for use in Debian, and have encountered the following build error when using github.com/shirou/gopsutil/v3 v3.22.10 (the current version of that library and recently updated in Debian):

# github.com/minio/madmin-go
src/github.com/minio/madmin-go/health.go:328:36: undefined: host.Warnings

It appears that commit https://github.com/shirou/gopsutil/commit/dbc0f20fe38841a0c1509de9dba5084b7c2c09d9 moved host/types.go -> internal/common/warnings.go.

Hi, First and foremost, thanks for providing this super useful pkg. I want to get the number of reads and writes of the bucket through the madmin.ServerInfo interface，But I don't know which parameter I need."ReadVersion" always changes when I read and write files to the bucket, but other numbers don't change What can I do to get the results I want？

Version 2.0.0 switched the license over to AGPL-3, but a handful of files still list Apache-2 as their license.

For example, in cpu_linux.go:

//
// Copyright (c) 2015-2022 MinIO, Inc.
//
// This file is part of MinIO Object Storage stack
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as
// published by the Free Software Foundation, either version 3 of the
// License, or (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
//

//go:build linux
// +build linux

//
// MinIO Object Storage (c) 2021-2022 MinIO, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//      http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//

(You can find all such files via grep -ri apache.)

Is this an oversight, or are the affected files dual-licensed?

Signed-off-by: Ricardo Katz [email protected]

madmin-go does not support skipping insecure verification.

This PR adds support for skipping the tls verification when using madmin-go client.

Also a few other minor changes:

• simpler code for preparing CPU list
• few struct methods useful for health analyzer
• rename and export the health info version struct

Client functionality for the IDP admin APIs updated in https://github.com/minio/minio/pull/15896 - includes code to be compatible to previous version of released IDP API.

Add realtime metrics structs and add client that can be used for several metric types. Currently only scanner.

Refactors Trace options. Backwards compatible with older servers, but breaks API (struct changes). API breakage is to make TraceInfo fields optional for different types.

PR for minio, console & mc are ready and will be added when this is in.

For including the new perf test (speedtest) results in the output of mc support diag,

• Move the current perf related types (PerfInfo and others) to health-old.go
• Move the current HealthInfo struct also to health-old.go and rename it to HealthInfoV2
• Introduce new type SpeedTestResults to hold all speedtest results
• Introduce new HealthInfo which is same as the current one, except for using SpeedTestResults instead of PerfInfo
• Change names of perf tests So that when this mc is used with older versions of minio, it doesn't request for the old performance tests that are slow and are not very reliable.
• Bump up the health info version

Hi,

First and foremost, thanks for providing this super useful pkg.

If I follow the example you provided for adding a policy, which is the following:

policy, err := iampolicy.ParseConfig(strings.NewReader(`{"Version": "2012-10-17","Statement": [{"Action": ["s3:GetObject"],"Effect": "Allow","Resource": ["arn:aws:s3:::my-bucketname/*"],"Sid": ""}]}`))
if err != nil {
       log.Fatalln(err)
}
if err = madmClnt.AddCannedPolicy(context.Background(), "get-only", policy); err != nil {
       log.Fatalln(err)
}

The policy is given as Policy type whereas in the function implementation it is expecting []bytes:

func (adm *AdminClient) AddCannedPolicy(ctx context.Context, policyName string, policy []byte) error

I'm using the pkg github.com/minio/pkg/iam/policy to parse my policies, therefore I could notice that there is a function that I can use to convert back the policy to JSON in []bytes:

func (policy Policy) MarshalJSON() ([]byte, error) {
	if err := policy.isValid(); err != nil {
		return nil, err
	}

	// subtype to avoid recursive call to MarshalJSON()
	type subPolicy Policy
	return json.Marshal(subPolicy(policy))
}

However, when I try to use it I get this error:

policy.MarshalJSON undefined (type *iampolicy.Policy has no field or method MarshalJSON)

I relay on the policy pkg to do other actions (e.g isValid ); Therefore workarounds like creating directly the policy in []bytes format (see below) would not work for me.

    policyBytes := []byte(fmt.Sprintf(`{
        "Version": "2012-10-17",
        "Statement": [
         {
          "Effect": "Allow",
          "Action": [
           "s3:PutObject",
           "s3:GetObject",
           "s3:ListBucket"
          ],
          "Resource": [
           "arn:aws:s3:::%s/*"
          ]
         }
        ]
       }`, "bucket"))

Any help on how to convert Policy to []bytes?

Regards, Salim.

go 1.17
github.com/minio/madmin-go v1.1.23
github.com/minio/pkg v1.1.11

In fact behaviour of this method depends if user already exists or not, and in 1st case it will just update his secret key (and status). Thus, for more transparency would be better maybe to change it to e.g CreateOrUpdateUser? Or at least add additional comment up of the method definition.

https://github.com/minio/madmin-go/blob/f1dab86fb8ecfae0d83c9a7cc0f4c16303ae4c58/user-commands.go#L266

Hi,

In the UI, when creating a personal service account, the option exists to generate the session policy document, edit it, and attach it to the service account.

We have MinIO authentication and authorisation using LDAP and policies on LDAP groups. That works nicely.

Sometimes users join new LDAP groups and therefore get updated session policies. For automation purposes (secret rotation) we would like to update existing service accounts (system wide but user specific) with new policy documents. These new policy documents should be generated by MinIO similar to what the UI does. That way we can update and/or rotate existing/new service accounts with the current complete policy document.

However, using minio admin go client: mc admin user policy myminio "CN=Hartwig\, Jonas (cvv556),OU=...." I get the following error: mc: <ERROR> Unable to fetch user policy document. Policy not found for user CN=Hartwig\, Jonas (cvv556),OU=.... When using the minio admin go client, create a service account without policy attached, retrieve it and investigate it, it is shown as implied/empty policy. That also makes sense.

Another use case for admins would be to debug the policies. The setup might be quite complex of policies assigned to users and groups. The resulting policy document might be conflicting. It would be helpful to understand a user policy in case of complains.

Now here is what I need: How can I get this implied policy document?

Regards