An entity framework for Go

This is a basic start, a lot of cut and paste but it does pass some very basic tests now. I still have more work to do but I was hoping we could discuss some of it as it arrives, so here's the PR.

This is a feature that I have wanted to add for a long time to ent, and will be happy to get feedback, suggestion for improvements or just hear your thoughts before I create a proper PR for it.

The idea is instead of adding a completely new API for adding fields for edges (columns for join tables), configure their indexes, hooks, privacy, etc, or even support multi-column PKs, we'll add 2 new small changes for the ent/schema for supporting all those issues.

field.ID Annotation

A new field.ID option will be added to the schema/field package, and this will allow configuring multi-column PKs, but more than that, it will make it possible to configure relation/edge schema manually. For example:

// UserGroup defines the UserGroup relation schema.
type UserGroup struct {
	ent.Schema
}

func (UserGroup) Annotations() []schema.Annotation {
	return []schema.Annotation{
		// This will generate the following struct:
		//
		//	type UserGroupID struct {
		//		UserID, GroupID int
		//	}
		//
		field.ID("user_id", "group_id"),
	}
}

func (UserGroup) Fields() []ent.Field {
	return []ent.Field{
		field.Int("user_id"),
		field.Int("group_id"),
		field.Time("created_at").
			Immutable().
			Default(time.Now),
	}
}

func (UserGroup) Edges() []ent.Edge {
	return []ent.Edge{
		edge.To("user", User.Type).
			Unique().
			Field("user_id"),
		edge.To("group", Group.Type).
			Unique().
			Field("group_id"),
	}
}

Indexes, hooks, privacy, and all other options will also be available for this type of schema, but in order to add a user to a group, the flow is a bit longer:

a8m := client.User.Create().SetName(..).SaveX(ctx)
hub := client.Group.Create().SetName(..).SaveX(ctx)
// Instead of calling hub.AddGroups(a8m), the API is:
client.UserGroup.Create().SetUserID(a8m.ID).SetGroupID(hub.ID).ExecX(ctx)
client.UserGroup.GetX(ctx, ent.UserGroupID{UserID: a8m.ID, GroupID: hub.ID})

We can bypass this long flow by providing an additional config option for edges:

The edge.Through Option

The Through option is supported by other frameworks and has already been proposed here before. The idea is to allow schemas that use the relation/edge-schema to CRUD directly their relations. However, I have an open question (atm) regarding the design of the generated code.

The configuration looks as follows:

func (Group) Edges() []ent.Edge {
	return []ent.Edge{
		edge.To("users", User.Type).
			Through(UserGroup.Type),
	}
}

And the generated API is as follows:

a8m := client.User.Create().SetName(..).SaveX(ctx)
hub := client.Group.Create().SetName(..).AddUsers(a8m).SaveX(ctx)

An open question I have is on how do we set/update relation-schema fields in case they don't have default values or hooks configured on their ent/schema.

Thoughts?

As part of the effort started by @cliedeman and @giautm on https://github.com/ent/contrib/pull/235, I propose breaking it into smaller tasks and pushing it upstream gradually. We should start with the most common and requested option - generating GraphQL objects from an Ent schema. I suggest starting with the following but would love to hear if you think we should add more.

Stage 1:

Schema-levlel annotation

• Generate basic GraphQL objects (type T), allow configuring its name, directives, implemented interfaces, and its exported fields.
• Generate Relay Connection objects and the rest of the types supported by https://entgo.io/docs/tutorial-todo-gql-paginate/
• Auto-update gqlgen config file

Project-levlel annotation (entc.go)

• Allow generating types like interfaces or types that are globally used and not specific to a schema.

Stage 2:

• Allow configuring mutations (actions) and their inputs on the schema-level and generate both gql and ent code for them.
• Generate queries.

Implementation ideas

I really like the entgql.RelayConnection option defined in https://github.com/ent/contrib/pull/235, but for the types, I think we should make it more customized.

1. Auto-generate GraphQL and Relay types, but skip fields that are marked with Skip.

func (Pet) Annotations() []schema.Annotation {
	return []schema.Annotation{
		entgql.Type()
		entgql.RelayConnection(),
	}
}

func (Pet) Fields() []ent.Field {
	return []ent.Field{
		// ...
		field.String("name").
			Annotation(entgql.Skip()),
	}
}

2. More control on the generated model and fields.

func (Pet) Annotations() []schema.Annotation {
	return []schema.Annotation{
		entgql.Type("GQLType").
			Directive(...).
			Comment(...).
			Implemented(...)
		entgql.RelayConnection(),
	}
}

func (Pet) Fields() []ent.Field {
	return []ent.Field{
		// ...
		field.String("name").
			Annotation(
				entgql.Field("gql_name").
					Comment(...),
			),
	}
}

3. Allow free mapping.

func (Pet) Annotations() []schema.Annotation {
	return []schema.Annotation{
		entgql.Type("GQLType").
			Fields(
				entgql.MapField("gql_field1", "<edge or field name>").
					Comment(...),
				entgql.MapField("gql_field2", "<edge or field name>").
					Directive(...),
				// ...
			),
		entgql.RelayConnection(),
	}
}

I have drafts for stage-2, but let's focus on stage-1 first. Thoughts?

cc @crossworth, @giautm, @cliedeman, @rotemtam, @masseelch, @yonidavidson, @maaft

Is there a way to redeclare the id column as a custom string type?

If I put the following in my schema:

field.String("id").Unique()

The generated code ends up broken because it creates two declarations of "id", but furthermore it ignores the fact that it's a string, and declares it as an integer instead.

This might be related to the UUID discussions, but it would be great if we can define our own custom primary keys as strings that we provide at runtime.

Update

Please see https://entgo.io/docs/schema-edges/#edge-field for full docs.

Add an API for loading and getting the FKs of ent models, and gate it with feature-flag.

This change adds a new method on each <T>Query (with FKs on its table) named WithFK() for loading the FKs together with all other fields. However, if users want to select specific fields, they can use Select(...) for selecting any field (including FK). For example:

pets, err = client.Pet.Query().
	WithFKs().
	All(ctx)

pets, err = client.Pet.Query().
	Select(pet.FieldName, pet.ForeignKeys...).
	All(ctx)

Also, for each field-edge (an edge with FK), we generate a new method (e.g. OwnerID() (int, error)), that returns the value of the foreign-key, or an error if it wasn't loaded on query or wasn't found (was NULL in the database). For example:

owner, err := luna.OwnerID()

Next steps (in future PRs) are to add the Select option also to mutations (e.g. Pet.Update()) and predicates for simplifying queries:

pets, err := client.Pet.Query().
	Where(pet.OwnerID(id)).
	All(ctx)

Please share you feedback. Thanks cc @aight8 @errorhandler @Siceberg @rubensayshi

Is it possible to have atomic upsert functionality similar to postresql's INSERT ON CONFLICT or mysql's INSERT ON DUPLICATE KEY UPDATE statement? Use case is to save roundtrips on remote databases and to minimize lines of code.

Summary: Pull Request resolved: https://github.com/facebookexternal/fbc/pull/1527

See https://circleci.com/blog/level-up-go-test-with-gotestsum/ for more info

Differential Revision: D17761305

I don't figure out yet but latest package does not work on my environment.

• [x] The issue is present in the latest release.
• [x] I have searched the issues of this repository and believe that this is not a duplicate.

Current Behavior 😯

ent generate ent

output

internal error: package "context" without types was imported from "entgo.io/ent"

Expected Behavior 🤔

finish with successfully

Steps to Reproduce 🕹

Steps:

1. just do ent generate ent

Your Environment 🌎

| Tech | Version | | ----------- | ------- | | Go | go version devel go1.18-17980dff36 Wed Nov 10 05:08:25 2021 +0000 windows/amd64 | | Ent | latest commit | | Database | SQLite3 | | Driver | https://github.com/mattn/go-sqlite3 |

Super excited to see ent support gqlgen (although I understand it's still early on :))

I was giving it a spin this evening and noticed when trying to create a model using uuid.UUID as the ID type, I got a bunch of type errors in some of the generated code.

ent/node.go:51:3: cannot use c.ID (type uuid.UUID) as type int in field value
ent/node.go:86:3: cannot use u.ID (type uuid.UUID) as type int in field value
ent/node.go:190:21: cannot use id (type int) as type uuid.UUID in argument to testmodel1.ID
ent/node.go:199:17: cannot use id (type int) as type uuid.UUID in argument to testmodel2.ID
ent/pagination.go:450:18: cannot use c.ID (type uuid.UUID) as type int in field value
ent/pagination.go:669:18: cannot use u.ID (type uuid.UUID) as type int in field value

So I changed the generator based from the documentation to use a custom Field TypeInfo to TypeUUID

// +build ignore

package main

import (
	"log"

	"github.com/facebook/ent/entc"
	"github.com/facebook/ent/entc/gen"
	"github.com/facebook/ent/schema/field"
	"github.com/facebookincubator/ent-contrib/entgql"
)

func main() {
	err := entc.Generate("./schema", &gen.Config{
		Templates: entgql.AllTemplates,
		IDType: &field.TypeInfo{
			Type: field.TypeUUID,
		},
	})
	if err != nil {
		log.Fatalf("running ent codegen: %v", err)
	}
}

Which got me a little bit further!

ent/node.go:180:26: cannot use id (type [16]byte) as type string in argument to strconv.Atoi

Looks like the culprit is located here. As the type of id argument in Noder is [16]byte, this wont work but I'm wondering if there's an easy way to set this as uuid.UUID and use a Stringer interface to call Atoi?

Thanks!

I have roughly the schema below, and on the second pass of Schema.Create, it gives this stack trace. I notice on the documentation near the trace it says sqlite3 can't be used in "append-only" mode because it's "just for testing". Is that the current state of things? If so I can just change my database targets.

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x50 pc=0x957333]

goroutine 1 [running]:
github.com/facebookincubator/ent/dialect/sql/schema.(*Migrate).changeSet(0xc00011f740, 0x0, 0x10b6c40, 0xc8f660, 0xc00015a860, 0xbb0ea2)
        /usergo/pkg/mod/github.com/facebookincubator/[email protected]/dialect/sql/schema/migrate.go:241 +0x63
github.com/facebookincubator/ent/dialect/sql/schema.(*Migrate).create(0xc00011f740, 0xc8f460, 0xc0000a0000, 0xc8f660, 0xc00015a860, 0x10b0920, 0x3, 0x3, 0xbd9320, 0xc00011f740)
        /usergo/pkg/mod/github.com/facebookincubator/[email protected]/dialect/sql/schema/migrate.go:121 +0x1b6
github.com/facebookincubator/ent/dialect/sql/schema.(*Migrate).Create(0xc00011f740, 0xc8f460, 0xc0000a0000, 0x10b0920, 0x3, 0x3, 0x0, 0x0)
        /usergo/pkg/mod/github.com/facebookincubator/[email protected]/dialect/sql/schema/migrate.go:104 +0x11f
github.com/tinyci/services/service/usersvc/ent/migrate.(*Schema).Create(0xc0000ab6c0, 0xc8f460, 0xc0000a0000, 0x0, 0x0, 0x0, 0x0, 0xc0000d0d80)
        /usergo/src/github.com/tinyci/services/service/usersvc/ent/migrate/migrate.go:48 +0x12a

Schema:

// User holds the schema definition for the User entity.
type User struct {
  ent.Schema
}

// Fields of the User.
func (User) Fields() []ent.Field {
  return []ent.Field{
    field.String("username").Immutable().Unique(),
    field.String("password"),
    field.Time("created_at").Immutable().Default(time.Now),
    field.Time("updated_at").Default(time.Now),
  }
}

// Edges of the User.
func (User) Edges() []ent.Edge {
  return []ent.Edge{
    edge.To("capabilities", Capability.Type),
    edge.To("tokens", AuthToken.Type),
  }
}

// Capability holds the schema definition for the Capabilities entity.
type Capability struct {
  ent.Schema
}

// Fields of the Capabilities.
func (Capability) Fields() []ent.Field {
  return []ent.Field{
    field.String("capability").Immutable().Unique(),
  }
}

// Edges of the Capabilities.
func (Capability) Edges() []ent.Edge {
  return []ent.Edge{
    edge.To("users", User.Type).Unique(),
  }
}


// AuthToken holds the schema definition for the AuthToken entity.
type AuthToken struct {
  ent.Schema
}

// Fields of the AuthToken.
func (AuthToken) Fields() []ent.Field {
  return []ent.Field{
    field.String("service").Immutable(),
    field.String("token").Sensitive(),
    field.Time("created_at").Immutable().Default(time.Now),
    field.Time("expires_at").Immutable().Nillable().Optional(),
  }
}

// Edges of the AuthToken.
func (AuthToken) Edges() []ent.Edge {
  return []ent.Edge{
    edge.To("user", User.Type).Required().Unique(),
  }
}

Can we generate the versioned migration files not in main?

for example:

func Open(ctx context.Context, databaseUrl string) *ent.Client {
	db, err := sql.Open("pgx", databaseUrl)
	if err != nil {
		log.Fatalf("failed to open the database", err)
	}

	// Create an ent.Driver from `db`.
	drv := entsql.OpenDB(dialect.Postgres, db)
	client := ent.NewClient(ent.Driver(drv))

	dir, err := migrate.NewLocalDir("migrations")
	if err != nil {
		log.Fatalf("failed creating atlas migration directory: %v", err)
	}
	// Write migration diff.
	err = client.Schema.Diff(ctx, schema.WithDir(dir))
	if err != nil {
		log.Fatalf("failed creating schema resources: %v", err)
	}

	return ent.NewClient(ent.Driver(drv))
}

I noticed that on Graphql when using or predicate and put in or section ID field, it's like ID field is skipping. I only checked Graphql, so maybe on code all is right.

• [x] The issue is present in the latest release.
• [x] I have searched the issues of this repository and believe that this is not a duplicate.

Current Behavior 😯

This example is getting <=0 result

{
  "where": {
    "or": {
      "someStr": "0208e28b-cbfd-5a71-bb06-0e9d79d327ed",
      "someStr2": "0208e28b-cbfd-5a71-bb06-0e9d79d327ed",
      "id": "0208e28b-cbfd-5a71-bb06-0e9d79d327ed"
    }
  }
}

This example is getting >=1 result

{
  "where": {
      "id": "0208e28b-cbfd-5a71-bb06-0e9d79d327ed"
  }
}

Expected Behavior 🤔

So or field should support ID fields too.

Steps to Reproduce 🕹

Steps:

1. Run any query with or predicate
2. Use ID field on or section
3. Compare result with direct ID field filter result

Your Environment 🌎

| Tech | Version | | ----------- | ------- | | Go | 1.19 | | Ent | master | | Database | Postgresql |

I noticed that predicates for slice of strings not implemented on GraphQL

• [ ] I have searched the issues of this repository and believe that this is not a duplicate.

Summary 💡

So will be great to have all intuitive predicates for array fields (in, notIn, eq, e.t.c)

Motivation 🔦

Requesting to make EntRefine search more powerful

Title: Add gRPC async stream support to ent

• [x] I have searched the issues of this repository and believe that this is not a duplicate.

Summary

ent should support gRPC async streams to allow users to take advantage of the performance, scalability, and flexibility offered by this feature when building applications that use ent as their database. This would also make it easier to integrate ent with other systems that use gRPC.

Motivation

gRPC async streams allow clients and servers to send and receive an unbounded number of messages over a single connection. Adding support for gRPC async streams to ent would provide benefits to users, including improved performance, increased scalability, and greater flexibility when exchanging data between the client and server. The lack of this feature has made it more difficult to build applications that use ent as their database and to integrate ent with other systems that use gRPC.

Hey, how to create a custom field for file handling. So that a User can upload a file via a generated gRPC service and the content of the file will be stored on a local file path. While a User create a get request on a field resource, he will get the content of the file with same metadata like name & hash.

As orientation I like the way, how Django did this: https://docs.djangoproject.com/en/3.2/_modules/django/db/models/fields/files/

Currently, I'm kind lost and do not know where to start. My end goal, would be to store the file on a S3 service.

Thanks for any help.

• [x] I have searched the issues of this repository and believe that this is not a duplicate.

Summary 💡

Suppose we have Groups of resources, where there are different entities for each resource type, such as ResourceA and ResourceB. Each resource type can have many groups, and each group can have many resources of any type.

I should be able to write a schema like this:

package schema

import (
	"entgo.io/ent"
	"entgo.io/ent/schema/edge"
	"entgo.io/ent/schema/field"
)

type Group struct {
	ent.Schema
}

func (Group) Edges() []ent.Edge {
	return []ent.Edge{
		edge.To("resourceAs", ResourceA.Type).Through("AConnections", Connection.Type),
		edge.To("resourceBs", ResourceB.Type).Through("BConnections", Connection.Type),
	}
}

type ResourceA struct {
	ent.Schema
}

func (ResourceA) Edges() []ent.Edge {
	return []ent.Edge{
		edge.From("groups", Group.Type).Ref("resourceAs").Through("AConnections", Connection.Type),
	}
}

type ResourceB struct {
	ent.Schema
}

func (ResourceB) Edges() []ent.Edge {
	return []ent.Edge{
		edge.From("groups", Group.Type).Ref("resourceBs").Through("BConnections", Connection.Type),
	}
}

type Connection struct {
	ent.Schema
}

func (Connection) Fields() []ent.Field {
	return []ent.Field{
		field.Uint("group_id"),
		field.Uint("resourcea_id").Optional().Nillable(),
		field.Uint("resourceb_id").Optional().Nillable(),
	}
}

func (Connection) Edges() []ent.Edge {
	return []ent.Edge{
		edge.To("group", Group.Type).Required().Unique().Field("group_id"),
		edge.To("resourceA", ResourceA.Type).Unique().Field("resourcea_id"), // only one resource edge will be not null
		edge.To("resourceB", ResourceB.Type).Unique().Field("resourceb_id"), // only one resource edge will be not null
	}
}

Note that a Connection must have a non-null Group edge, but will only have one non-null resource edge. So a Group with one ResourceA and one ResourceB will need two Connection entities, one for connecting A and one for connecting B.

The above schema will trigger a panic during generation because Connection is used in more than one EdgeSchema. If I instead only try to create the EdgeSchema for ResourceA, I get an error that the resourceA edge of Connection must be Required(), which doesn't fit how we actually want to use this table.

Motivation 🔦

This would be more of a convenience than anything. The existing M2M supports looks great and it would be nice if it could be used on a larger variety of M2M implementations.

For now, I can instead define this with 2 edge hops Group -> Connection -> ResourceA, but it would be nice if I could
treat this as a single edge Group -> ResourceAs.

Hey :-) I´ve researched a while now, but it seems i dont really get it. I have a column defined as BIGSERIAL. I dont want to set this value on Create() (since it should auto increment), so i want to make it optional.

field.Int("insertion_order").
	Optional().
	SchemaType(map[string]string{
		dialect.Postgres: postgres.TypeBigSerial,
}),

This results in an error: NOT NULL constraint is required for bigserial column

If i remove the optional,. i am forced to set this value, which i dont want, since it should auto increment.

On the other hand, i´m also not able to make an custom auto increment, since i see no option of creating sequences.

There is just one line i have to remove from code generation, then the example (without the optional) above is working:

if _, ok := pec.mutation.InsertionOrder(); !ok {
      return &ValidationError{Name: "insertion_order", err: errors.New(`ent: missing required field "PendingEvent.insertion_order"`)}
}

Maybe there is someone who can give me some hints on how to work with such a case :-)