Kubernetes GraphQL Dynamic Query in Go | by Stefanie Lai | Jun, 2022

Play some magic with client-go DiscoveryClient


In the first episode of Kubernetes GraphQL Query in Go, we built the server code using GraphQL to query Pods in a cluster. However, such queries are poor in flexibility and scalability since the definitions of the related graphql.Feilds are all hard-coded, which needs refactoring.

So here comes the episode two of Dynamic Query, using client-go DiscoveryClient to flexibly define the GraphQL query structure.

It is known to most Kubernetes users that Kubernetes resource objects like Pod and Deployment are all of Kubernetes native resource types, and essentially a Kubernetes native Operator is implemented in the form of CRD + Controller like those user-customized Operators. The only difference is the native resource types are scattered in different packages in Kubernetes source code.

How Operator processes
  • The user submits the CRD-definition-abiding YAML to the cluster.
  • Kubernetes APIServer stores resources in etcd based on the relevant CRD types.
  • The CRD-corresponded Controller finds the resources to be processed and performs the reconciliation process, including updating the resource status.
  • Save the latest resource status to etcd.

It is easy to conclude that CRD is the schema corresponding to the Kubernetes resource type, just as the DDL statement to a database table, or the XSD file to an XML file.

When building a GraphQL query corresponding to all resource types in the cluster, the first difficulty that confronts us is how to get the schema of all the resource types.

Kubectl command

kubectl command is always the first approach we would try in Kubernetes-related queries.

All resource types in the current cluster can be returned with kubectl get api-resources, such as Pod, Deployment, etc.


But not the case when running kubectl get crd that we see no return of native resource type like pod.

We definitely need to seek a supplementary to find the schema of the native resource types, and let’s try some.

  • Some YAML validator tools like kubeval, which points to some GitHub repositories containing schemas of various Kubernetes versions, such as yannh/kubernetes-json-schema. These GitHub schemas also support obtaining the JSON version schema via the corresponding URL, such as the pod schema for version 1.22.8.
  • kubectl explain command, which can print the schema of the corresponding type.

However, both of the above methods have hidden defects: The first one is limited by versions, and code upgrading is needed once the cluster upgrades; The second can only return the schema of the searched type, but not those of the nested type, and is not JSON-format supportive.

  • Discovery API, is the real thing I finally find that can sweep off the tricky issue.

The discovery package is used to discover APIs supported by a Kubernetes API server.

OpenAPISchema fetches the open api v2 schema using a rest client and parses the proto.

The OpenAPISchema API in DiscoveryClient returns schema definitions for all types defined in the cluster. You cannot fully feel this API’s powerfulness from its brief introduction, especially when no relevant example is displayed in the client-go.

But trying more will never disappoint you. In the beginning, I only tried to use this API to get the definition of CRD under the guide of Kubernetes documentation “CRD is an optional OpenAPI v3 based validation schema.” Then I was lucky to discover that it can return all schema definitions in the cluster, including native types.

Kubernetes API

Let’s dig deeper and see why OpenAPISchema function can return all schemas?

Nothing magical, but by simply calling the Kubernetes /openapi/v2 API.

d.restClient.Get().AbsPath(“/openapi/v2”).SetHeader(“Accept”, openAPIV2mimePb).Do(context.TODO()).Raw()

Turn back to the previous kubectl explain command, add -v flag, and run it again.

kubectl explain pods -v 9 displays

So as is seen, it is OpenApi that processes behind the explain command, but receives data in protobuf format, and offers the users higher readability.

We can also try calling OpenApi in CLI, and the following command is for GKE.

curl -X GET https://{ip}/openapi/v2 --header "Authorization: Bearer $(gcloud auth application-default print-access-token)" --insecure

We need to fully understand the information we got before we put it to good use.

Exporting the output of curl command to a file, we can see that all we get is the analysis of each API in Kubernetes, just like the swagger API documentation. And roll down to the bottom to see the definitions, where the definitions of all fields for all resource types are shown.

The Pod definition would be long, so take the resources field below as an example.

client-go has parsed the schema into an openapi_v2 Document, which is ready to be applied to development.

But one issue should not be ignored: With all resource types returned, it is hard to distinguish the real top-level resource types, for example, we don’t want users to query PodAffinity, but Pods only.

DiscoveryClient also provides API ServerGroups and ServerResourcesForGroupVersion to help build the GKV(GroupKindVersion) needed.

Now we can finally get down to parsing schema.

  • Locate the resource type to parse
  • Find the fields in the type.
  • Build the corresponding graphql.Fields.

The key to this step is to build the connection between the corresponding Kubernetes Go types and the GraphQL types via graphqlType function.

  • For the common type, connect straightforwardly. Such as
// a TypeItem from the schema
switch openApiType {
case "boolean":
return graphql.Boolean
case "integer":
return graphql.Int
case "number":
return graphql.Float
case "string":
return graphql.String
  • For the object type, build the object type in GraphQL after recursive parsing.
case "object":
objectFields := parse(document, schema)
return graphql.NewObject(graphql.ObjectConfig{
Name: rt.Kind + "Props",
Description: schema.GetDescription(),
Fields: objectFields,
  • For the array type, parse the GraphQL type corresponding to the first element, and then combine it into the GraphQL List type.
case "array":
iType = graphqlType(docuemnt, possible_type, schema.GetItems().GetSchema()[0])
return graphql.NewList(iTypes)

After the loop, we get the GraphQL Fields corresponding to all resource types in the cluster, which can be further combined into a GraphQL Object.

Now, let’s do a quick test in the localhost:8080/graphql. Rerun the program, and try to fetch the information about the PrometheusRule resources. Excellent! GraphQL not only returns the relevant information but also relevant prompts when editing the request.

GraphQL has been introduced for 10 years, but it is not so widely adopted as REST. It’s hard to explain why, just as you cannot figure out why many Java programs are still staying with Java8, or even Java 7.

But I have faith in GraphQL. It has huge advantages over REST in many aspects, such as it greatly simplifies the frontend-backend API design interaction, and improves developing efficiency. And being employed by more and more Internet companies, it will become the development standard in the future.

Thanks for reading!

discovery package — — Go Packages

News Credit

%d bloggers like this: