Caddy 2 port

[BTBurke]

@mholt has announced that Caddy 2 may hit stable by early next year. Based on a quick glance at the new docs, there are lots of changes to how plugins (soon to be called modules) work, and it will likely require a complete rewrite to support JWT auth in Caddy 2. I plan to post a deprecation notice on the readme so people know that no further development will happen on this plugin unless it's to address a security vulnerability.

I haven't decided whether I'll port this to the new module system as I rarely use JWT anymore. I think paseto has a better design and less foot guns if you are starting from scratch on a new application.

This issue is meant as a discussion area for a potential port to Caddy v2. For anyone considering an auth module for Caddy, I'd offer the following ideas based on several years of experience hearing about various use cases.

Eliminate all Caddyfile-style configuration directives that express auth logic

One of the major drawbacks of v1-style plugins was the use of the Caddyfile for expressing auth logic. While the system of allow user bob worked, it's awkward to use a configuration language to express logic. This is a recurring problem with projects that use config files to express branching logic that depends on external state (e.g., Ansible, Kubernetes Helm templates, etc.) where if-then-else logic is bolted on to a declarative config file format. This can be solved by using Starlark as suggested below, but there may be other ways to move this logic outside Caddy configuration and into an environment that is more suitable to scripting. One option is to delegate auth decisions to a separate microservice as in caddy-extauth, so that authorization can be controlled by the user in any language, but easily integrated as a middleware-type plugin.

The new module should take advantage of the integrated Starlark scripting capability to provide a platform for users to express advanced auth flows.

Starlark has a python-like syntax that would allow someone to express all kinds of auth flows and special cases that are awkwardly handled as edge cases in this plugin. Things like extracting tokens from oddly named cookies, stripping claim prefixes, and returning non-standard HTTP responses would be relatively easy in Starlark and keep these edge cases out of the module code.

token = token_from_cookie("my-auth-token")
secret = secret_from_env("JWT_SECRET")

if claim(token, "user") == "bob" and valid(token, HS256, secret):
     return ok()
else:
     return redirect(301, "https://mysite.com/login")

The starlark runtime can be injected with custom functions (implemented in Go), such as valid(token, algorithm, secret) and claim(token, field), that would extract claims and validate tokens, return various HTTP responses, set headers, allow fine grained auth rules for different paths, etc. so that nearly any use case can be under the control of the user while keeping the module code free of special cases.

Address drawbacks of delegating auth decisions to user scripts

One downside of the ideas above is that it would be a lot easier for a user to shoot themselves in the foot unless they are familiar with the design errors in JWT. There are several known vulnerabilities that arise from the JWT specification, like attacks based on monkeying with the alg header field in order to control how JWT validation is applied.

Care should be taken in what functions are supplied to the user so that the design of the API prevents them from insecurely validating tokens.

[mholt]

Great, I'm glad you bring this up! I think you raise some really good points.

Ideally, the logic will be abstracted away from users enough that they can't make security logic errors. That's kind of the Caddy way.

One possibility is to sacrifice some flexibility for the most common, simple configs, that can be expressed in a declarative syntax. Then allow Starlark scripting to handle more complex, imperative logic. But maybe the Starlark APIs can be designed in a way that it still provides the flexibility users need, without the possibility of making security mistakes.

Also, it would be preferable if it was implemented as native Go code, especially when the need is common enough. Go code will always be faster than an embedded scripting language, and easier to maintain. (Although Starlark is quite fast.)

Caddy 2 has an HTTP authentication module which, I hope, can help provide authentication for HTTP requests by any kind of provider. Right now, the only provider is "basicauth" which uses HTTP Basic Authentication. But I hope that it will one day support plugins that provide JWT, OAuth2, and other methods.

Auth providers implement this interface:

// Authenticator is a type which can authenticate a request.
// If a request was not authenticated, it returns false. An
// error is only returned if authenticating the request fails
// for a technical reason (not for bad/missing credentials).
type Authenticator interface {
	Authenticate(http.ResponseWriter, *http.Request) (User, bool, error)
}

We're still in beta, though, so we can and should change this interface if this is not sufficient.

My hope for this authentication module is that you can provide 1 set of user accounts or credentials or whatever, and just plug in and configure various ways to authenticate them. I don't know if that's possible for all auth methods, but it sure would be nice, right? (i.e. one list of accounts that you can authenticate via basic auth, Oauth2, JWT, etc.)

Let me know what you think, or if this is helpful at all. Also if there are any changes to be made, now is the time!

[BTBurke]

Hey Matt, thanks for that info. I think it's definitely possible to implement that interface across a wide variety of auth methods. One change I'd like to see is to expand the definition of the User struct to allow passing additional metadata about the user that could then be acted on by subsequent middleware. This would be great for JWT, Paseto, and other formats that pass arbitrary claim information as part of the token.

For example, a metadata field as a json.RawMessage would help a lot in being able to chain an auth middleware with a subsequent Starlark or other middleware that would act on claim information passed along in the context.

type User struct {
   ID  string
   Metadata json.RawMessage
}

[jimjimovich]

First of all, thanks so much for the work you did on this plugin. It has been incredibly helpful to me over the last year!

I really hope that someone will be able to implement JWT for Caddy v2. It's a very common technology and your plugin made it incredibly easy to use.

[mholt]

One change I'd like to see is to expand the definition of the User struct to allow passing additional metadata about the user that could then be acted on by subsequent middleware. This would be great for JWT, Paseto, and other formats that pass arbitrary claim information as part of the token.

Absolutely -- I think another caddy 1 plugin (http.login, or loginsrv, I think?) has a User struct with a lot more fields.

And it's probably fine to add more strongly-typed fields too if that is more efficient, especially if they are useful/common in other auth methods.

I am mostly hoping the Authenticator interface is powerful and expressive enough to be compatible with all the various methods of authentication. If someone wanted to look into that for JWT (and others, if they want!), that'd be suuuper helpful. (I'm a bit swamped with other things right now, so. Community effort!)

[roblabla]

I did a very straightforward port in https://github.com/roblabla/caddy-jwt/tree/caddy2. It basically works exactly the same way (no fancy starlark integration or anything), but it:

1. Uses the Authenticator interface instead of ServeHTTP
2. Delegates all path matching to matcher (removes the path, allow_root, excludes options, as they can all be handled through a matcher combining path and not), simplifying the module a bit.

It's very untested so far. In fact, I had to disable all the unit tests because I couldn't figure out how to make them work yet (still figuring out caddy2). The next step for me is going to be porting loginsrv to caddy2. Once I've got both ported, I'll try getting unit tests back up.

Note to people who might be interested: To use it, you need to compile caddy with the following patch applied:

diff --git a/caddyconfig/httpcaddyfile/directives.go b/caddyconfig/httpcaddyfile/directives.go
index 035dcbec..9f86c35c 100644
--- a/caddyconfig/httpcaddyfile/directives.go
+++ b/caddyconfig/httpcaddyfile/directives.go
@@ -38,6 +38,7 @@ var directiveOrder = []string{
        "try_files",
 
        "basicauth",
+       "jwt",
        "header",
        "request_header",
        "encode",
diff --git a/modules/standard/import.go b/modules/standard/import.go
index 5ecfb4ac..48b1ce78 100644
--- a/modules/standard/import.go
+++ b/modules/standard/import.go
@@ -11,4 +11,5 @@ import (
        _ "github.com/caddyserver/caddy/v2/modules/caddytls/standardstek"
        _ "github.com/caddyserver/caddy/v2/modules/filestorage"
        _ "github.com/caddyserver/caddy/v2/modules/logging"
+       _ "github.com/roblabla/caddy-jwt/v4"
 )

and run go get github.com/roblabla/caddy-jwt/v4@caddy2

[mholt]

Nice start!

You can actually make it work without any changes to Caddy's source code.

Simply create your own main.go to add the import there instead, then use the order option in your Caddyfile:

{
    order jwt after basicauth
}

[roblabla]

I managed to get loginsrv ported over as well. However, I came across a fun little problem:

caddy-jwt and loginsrv have a cool "feature" where loginsrv will automatically generate a random jwt secret if none are set in the Caddyfile, and store it in the JWT_SECRET environment variable. When caddy-jwt later parses its own directive, it will use the secret found in the JWT_SECRET environment variable if none are configured.

Now, this is obviously ordering-dependent. The login directive must be parsed before the jwt directive for this to work. Which, obviously, doesn't really work out reliably well. I'm not sure how caddy parses the directive and how everything is ordered, but when I tried it, caddy-jwt didn't find any JWT_SECRET because it was parsed before loginsrv

So, anyone got ideas to make this work reliably? Is there a way to somehow make the two modules communicate a secret?

[magikstm]

Did you try these in your caddyfile:

{
   order jwt after basicauth
   order login before basicauth
}

It should allow them to be run in the correct order (login before jwt).

Ref: https://github.com/caddyserver/caddy/blob/master/caddyhttp/httpserver/plugin.go#L623

[roblabla]

From my understanding, order is an orthogonal concern: it will run the requests through the directives in the prescribed order. My problem has nothing to do with requests though, it's a kinda recey behavior at config parse/module load time. Basically, I need to ensure that loginsrv loads (not executes) before caddy-jwt.

Maybe order also affects other things than request (like Provision/Validate?), but at least the caddyfile parsing is not affected by order.

As a kinda hacky workaround, I'm currently working on lazily loading JWT_SECRET environment variable on first request in caddy-jwt. E.G. instead of grabbing the env vars directly in NewDefaultKeyBackends, have it unconditionally add an EnvVarKeyBackend, which will grab JWT_SECRET and cache its value when it executes its first request. Loginsrv would set the env variable in its provisioning step.

That would work, but is kinda ugly. I wonder if the module lifecycle interfaces like Provision/Validate are affected by order? If they are, I could simply make caddy-jwt fetch the JWT_SECRET in the provisioning step after loginsrv does it, but I'm doubtful that will work.

[mholt]

I wonder if the module lifecycle interfaces like Provision/Validate are affected by order?

Nope -- modules are loaded and provisioned in an order independent of the HTTP middleware/handler chain. But, don't expect one module to be provisioned before another, independent module, for example. If you absolutely need one to be provisioned before another, then perhaps one should wrap/encapsulate the other, and roll their Provision methods into one. Or something like that. It's actually not unheard of, even in Caddy's standard modules, for one module to literally embed another and simply wrap its behavior. If you have a tight coupling of two modules, they should probably be rolled into one or the logic/concerns should be split differently so that they can cleanly be two separate modules.

[roblabla]

Alright, I managed to get something that works while keeping the modules separate using the plan outlined above. It's hacky, but everything seems to be working properly so yay \o/.

Thinking about what mholt said, I think jwt should have the ability to load the loginsrv module (or the other way around) using a directive argument, automatically setting up the required configs. E.G. you could have

jwt /* login
# Expands to
jwt /* {
    token-string RANDOM_STRING
}
login {
    jwt-secret SAME_RANDOM_STRING
}

In practice, I think this can be achieved (still keeping the modules separate) by using RegisterDirective (instead of RegisterHandleDirective) and returning two routes in the parseCaddyFile, one for the jwt and one for the login. E.G.

var routes []httpcaddyfile.ConfigValue
// Parse loginsrv part of config
loginsrv_handler := loginsrv.CaddyHandler {}
routes := append(routes, c.NewRoute(nil, &loginsrv_handler)...)
// Parse jwt part of config
handler := jwt.Auth{}
routes := append(routes, c.NewRoute(nil, &jwt_handler)...)

@mholt does that sound reasonable? This would mean that this convenience is only usable from the caddyfile - JSON users would still have to setup both plugins separately, and make sure they pass the same token to both.

And an open questions: Should JWT embed LoginSrv or the other way around?

[mholt]

does that sound reasonable?

Yep, I think so anyway! In fact, that is how the php_fastcgi directive works (and a few others, though I can't recall atm). RegisterHandlerDirective is just convenient if you're making a single route/handler (which most directives do, but most directives are simpler).

You can also remove /* matchers from your Caddyfile, as that's the same as * which is the default anyway:

jwt login

However... before you go down that road...

I would strongly consider integrating with our existing authentication module -- as I mentioned in an earlier comment -- which is designed to support any mode of authentication using this Authenticator interface. If changes need to be made to the interface, let's do that ASAP before we leave beta. I'm willing to help with that.

I would love it if you can find a way for it to work with the existing auth module, before we leave beta. You can still make your own directives for the Caddyfile (this is what basicauth does, it integrates with the auth middleware but has its own directive).

[BTBurke]

Sorry I've been MIA for this discussion recently. I'd strongly prefer that we don't tightly couple this with loginsrv if I have to maintain it. It just makes maintaining both of those plugins more difficult than it should be and potentially more difficult to find security corner cases.

I'd also like it to integrate with whatever existing authentication hooks Caddy 2 has in it via the Authenticator interface. Since this is a security plugin, it needs a hard look at the tests to make sure it's still working as advertised. Honestly I haven't been following Caddy 2 development closely and I'm not up to speed on how the new modules work.