Rust Trait Lifetime Bounds

提问的故事 -- 废话哥 vs 高效哥

起因我折腾的时候, 升级了 zero-to-production 这个仓库的 tracing-bunyan-formatter 版本,

它当前依赖的是 0.2 版的 tracing-bunyan-formatter 和 tracing-subscriber :

tracing-subscriber = { version = "0.2.12", features = ["registry", "env-filter"] }
tracing-bunyan-formatter = "0.2.2"

但是由于 0.3 版的 tracing-subscriber 的 MakeWriter trait 签名改变了, 所以 0.3 版的 tracing-bunyan-formatter 的 new 方法的定义也改变了, 导致我直接编译不过.

0.2 的定义是这样的:

pub fn get_subscriber(
    name: String,
    env_filter: String,
    sink: impl MakeWriter + Send + Sync + 'static,
) -> impl Subscriber + Sync + Send {
    let env_filter =
        EnvFilter::try_from_default_env().unwrap_or_else(|_| EnvFilter::new(env_filter));
    let formatting_layer = BunyanFormattingLayer::new(name, sink);
    Registry::default()
        .with(env_filter)
        .with(JsonStorageLayer)
        .with(formatting_layer)
}

重点是这里的 sink: impl MakeWriter + Send + Sync + 'static

升级到 0.3 后就直接报错了:

error[E0106]: missing lifetime specifier
  --> src/telemetry.rs:18:16
   |
18 |     sink: impl MakeWriter + Send + Sync + 'static,
   |                ^^^^^^^^^^ expected named lifetime parameter
   |
help: consider introducing a named lifetime parameter
   |
15 ~ pub fn get_subscriber<'a>(
16 |     name: String,
17 |     env_filter: String,
18 ~     sink: impl MakeWriter<'a> + Send + Sync + 'static,
   |

For more information about this error, try `rustc --explain E0106`.
error: could not compile `zero2prod` due to previous error

这里的问题在于, 你如果一直试图按编译器的错误提示(tips)去修正错误, 是不可能完成这个fix的.

比如我按它上面的提示加个lifetime param <'a> :

pub fn get_subscriber<'a>(
    name: String,
    env_filter: String,
    sink: impl MakeWriter<'a> + Send + Sync + 'static,
) -> impl Subscriber + Sync + Send {
    let env_filter =
        EnvFilter::try_from_default_env().unwrap_or_else(|_| EnvFilter::new(env_filter));
    let formatting_layer = BunyanFormattingLayer::new(name, sink);
    Registry::default()
        .with(env_filter)
        .with(JsonStorageLayer)
        .with(formatting_layer)
}

然后你会发现这个报错越来越复杂了:

❯ cargo check
    Checking zero2prod v0.1.0 (/home/ttys3/repo/rust/zero-to-production)
error[E0277]: expected a `Fn<()>` closure, found `impl MakeWriter<'a> + Send + Sync + 'static`
  --> src/telemetry.rs:22:61
   |
22 |     let formatting_layer = BunyanFormattingLayer::new(name, sink);
   |                            --------------------------       ^^^^ expected an `Fn<()>` closure, found `impl MakeWriter<'a> + Send + Sync + 'static`
   |                            |
   |                            required by a bound introduced by this call
   |
   = note: wrap the `impl MakeWriter<'a> + Send + Sync + 'static` in a closure with no arguments: `|| { /* code */ }`
   = note: required because of the requirements on the impl of `for<'a> MakeWriter<'a>` for `impl MakeWriter<'a> + Send + Sync + 'static`
note: required by `BunyanFormattingLayer::<W>::new`
  --> .../tracing-bunyan-formatter-0.3.1/src/formatting_layer.rs:71:5
   |
71 |     pub fn new(name: String, make_writer: W) -> Self {
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
help: consider further restricting this bound
   |
18 |     sink: impl MakeWriter<'a> + Send + Sync + 'static + std::ops::Fn<()>,
   |                                                       ++++++++++++++++++

error[E0277]: expected a `Fn<()>` closure, found `impl MakeWriter<'a> + Send + Sync + 'static`
  --> src/telemetry.rs:22:28
   |
22 |     let formatting_layer = BunyanFormattingLayer::new(name, sink);
   |                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected an `Fn<()>` closure, found `impl MakeWriter<'a> + Send + Sync + 'static`
   |
   = note: wrap the `impl MakeWriter<'a> + Send + Sync + 'static` in a closure with no arguments: `|| { /* code */ }`
   = note: required because of the requirements on the impl of `for<'a> MakeWriter<'a>` for `impl MakeWriter<'a> + Send + Sync + 'static`
note: required by a bound in `BunyanFormattingLayer`
  --> .../tracing-bunyan-formatter-0.3.1/src/formatting_layer.rs:43:37
   |
43 | pub struct BunyanFormattingLayer<W: for<'a> MakeWriter<'a> + 'static> {
   |                                     ^^^^^^^^^^^^^^^^^^^^^^ required by this bound in `BunyanFormattingLayer`
help: consider further restricting this bound
   |
18 |     sink: impl MakeWriter<'a> + Send + Sync + 'static + std::ops::Fn<()>,
   |                                                       ++++++++++++++++++

error[E0277]: expected a `Fn<()>` closure, found `impl MakeWriter<'a> + Send + Sync + 'static`
  --> src/telemetry.rs:26:15
   |
26 |         .with(formatting_layer)
   |          ---- ^^^^^^^^^^^^^^^^ expected an `Fn<()>` closure, found `impl MakeWriter<'a> + Send + Sync + 'static`
   |          |
   |          required by a bound introduced by this call
   |
   = note: wrap the `impl MakeWriter<'a> + Send + Sync + 'static` in a closure with no arguments: `|| { /* code */ }`
   = note: required because of the requirements on the impl of `for<'a> MakeWriter<'a>` for `impl MakeWriter<'a> + Send + Sync + 'static`
   = note: required because of the requirements on the impl of `__tracing_subscriber_Layer<Layered<JsonStorageLayer, Layered<EnvFilter, Registry>>>` for `BunyanFormattingLayer<impl MakeWriter<'a> + Send + Sync + 'static>`
help: consider further restricting this bound
   |
18 |     sink: impl MakeWriter<'a> + Send + Sync + 'static + std::ops::Fn<()>,
   |                                                       ++++++++++++++++++

error[E0277]: expected a `Fn<()>` closure, found `impl MakeWriter<'a> + Send + Sync + 'static`
  --> src/telemetry.rs:19:6
   |
19 | ) -> impl Subscriber + Sync + Send {
   |      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected an `Fn<()>` closure, found `impl MakeWriter<'a> + Send + Sync + 'static`
   |
   = note: wrap the `impl MakeWriter<'a> + Send + Sync + 'static` in a closure with no arguments: `|| { /* code */ }`
   = note: required because of the requirements on the impl of `for<'a> MakeWriter<'a>` for `impl MakeWriter<'a> + Send + Sync + 'static`
   = note: required because of the requirements on the impl of `__tracing_subscriber_Layer<Layered<JsonStorageLayer, Layered<EnvFilter, Registry>>>` for `BunyanFormattingLayer<impl MakeWriter<'a> + Send + Sync + 'static>`
   = note: required because of the requirements on the impl of `tracing::Subscriber` for `Layered<BunyanFormattingLayer<impl MakeWriter<'a> + Send + Sync + 'static>, Layered<JsonStorageLayer, Layered<EnvFilter, Registry>>>`
help: consider further restricting this bound
   |
18 |     sink: impl MakeWriter<'a> + Send + Sync + 'static + std::ops::Fn<()>,
   |                                                       ++++++++++++++++++

For more information about this error, try `rustc --explain E0277`.
error: could not compile `zero2prod` due to 4 previous errors

不能继续这样下去了, 继续按它的提示 tips 修改, 永远也无法修复这个问题.

于是我去作者的这本书(我购买了他的电子书)的Discord群里问, 结果一周了还没人回答出我要的答案. 其中有一个回答了, 说了一堆废话最后的意思是: 你应该坚持使用 0.2 版的.

我就感觉这个人有点搞笑了... 纯浪费时间. 此为 "废话哥"

然后突然有一天, 来了另一个哥们 (此为"高效哥", "快手"-_-), 这哥们(不认识的 dude)二话不说直接把我拉进了一个 discoard thread, thread 标题也非常简单明了直接: "fixing get_subscriber"

这哥们直接说:

"MakeWriter现在需要一个lifetime我搞不定."
@asonix 好像上次更新这玩意是你弄的. 也许你可以帮我看下怎么搞定这个升级? 我无法弄明白怎么在这添加一个带lifetime的参数: https://github.com/LukeMathWalker/zero-to-production/blob/42d4f6a024fda2e7bc277679a595e3edfa2cb6c9/src/telemetry.rs#L18

然后 asonix 这哥们居然很快现身了. 回答道:

你可能需要放弃 impl MakeWriter 这种形式，转而使用泛型参数。你需要 for<'a> MakeWriter<'a> 约束所以你最终会得到

fn my_fn<M>(sink: M) -> impl Subscriber
where
    M: for<'a> MakeWriter<'a> + 'static,
{
  // ... 
}

dude: "Oh, wow, 多谢! 我还没在 Rust 里面用过这个 for 语法"
asonix: 是的，for 语法有点奇怪，但它用于表示“对于任何给定的生命周期都是如此”，而不是将约束限制为特定的生命周期，这很有用

最后, 这哥们 at 了一下我,

"来啦, 答案在这"

整个来回不超过6句话, 问题已经解决了.

于是我又回过头看 "提问的智慧"

发现里面真的有一句 "话不在多而在精".

所以, 其实这么多年了, 我还只掌握了这个文档的皮毛.

我在提问的时候, 提供了过多的无关信息(错误日志, 上下文等), 从而导致人们不想看, 因为没人想看一堆乱七八糟的东西. 大家都喜欢简单.
我没有以"最高效" 的方式提问.

这个哥们的提问为什么有如此高效的得到准确的答案? 原因是, 他是动过脑子的, 不是无脑地在群里直接提问. 而是有针对的 at 了相关作者.

比如这个例子里面, 使用到了 `tracing-bunyan-formatter 这个 crate, 通过一些基本的观察技能, 不难发现这个 commit 的作者正是 asonix:

https://github.com/LukeMathWalker/tracing-bunyan-formatter/commit/7acebe58a5ae3079e938a9a0a6dab1a7b5226692

一般人喜欢用相同的昵称, 根据这一点, 这哥们直接在 discord 群里 at 了 asonix, 并且以建立 thread 的方式, 相当于单独拉小黑屋讨论了.

为什么?

所以正确的参数是啥? 是这样:

pub fn get_subscriber<W>(
    name: String,
    env_filter: String,
    sink: W,
) -> impl Subscriber + Sync + Send
    where
        W: for<'a> MakeWriter<'a> + Sync + Send +  'static {
    let env_filter =
        EnvFilter::try_from_default_env().unwrap_or_else(|_| EnvFilter::new(env_filter));
    let formatting_layer = BunyanFormattingLayer::new(name, sink);
    Registry::default()
        .with(env_filter)
        .with(JsonStorageLayer)
        .with(formatting_layer)
}

impl MakeWriter + Send + Sync + 'static 要换成泛型参数, 然后给泛型参数增加约束.

所以 pub fn get_subscriber 变成 pub fn get_subscriber<W>

sink: impl MakeWriter + Send + Sync + 'static 变成 sink: W, 然后增加约束:

    where
        W: for<'a> MakeWriter<'a> + Sync + Send +  'static

我们看下 tracing-subscriber crate tracing_subscriber::fmt::MakeWriter 这个 trait 的变化, 其实也能找到模式.

0.2 版的时候是这样:

https://docs.rs/tracing-subscriber/0.2.25/tracing_subscriber/fmt/trait.MakeWriter.html

pub trait MakeWriter {
    type Writer: Write;
    fn make_writer(&self) -> Self::Writer;

    fn make_writer_for(&self, meta: &Metadata<'_>) -> Self::Writer { ... }
}

0.3 版就大变样了: https://docs.rs/tracing-subscriber/0.3.3/tracing_subscriber/fmt/trait.MakeWriter.html

pub trait MakeWriter<'a> {
    type Writer: Write;
    fn make_writer(&'a self) -> Self::Writer;

    fn make_writer_for(&'a self, meta: &Metadata<'_>) -> Self::Writer { ... }
}

那么, 原来的不是挺好的吗? 为什么要做这种改变?

我们去看一下 0.3.0 版的更新记录: https://github.com/tokio-rs/tracing/releases/tag/tracing-subscriber-0.3.0 果然发现这么一条:

fmt: Added a lifetime parameter to the MakeWriter trait, allowing it to return a borrowed writer. This enables implementations of MakeWriter for types such as Mutex<T: io::Write> and std::fs::File. (#781)

在 #781 https://github.com/tokio-rs/tracing/pull/781 这个 PR 里面, 作者详细地解释了做这种 breaking change 的必要性.

Motivation

Currently, the tracing-subscriber crate has the MakeWriter trait for customizing the io writer used by fmt. This trait is necessary (rather than simply using a Write instance) because the default implementation performs the IO on the thread where an event was recorded, meaning that a separate writer needs to be acquired by each thread (either by calling a function like io::stdout, by locking a shared Write instance, etc).

Right now there is a blanket impl for Fn() -> T where T: Write. This works fine with functions like io::stdout. However, the other common case for this trait is locking a shared writer.

Therefore, it makes sense to see an implementation like this:

impl<'a, W: io::Write> MakeWriter for Mutex<W>
where
    W: io::Write,
{
    type Writer = MutexWriter<'a, W>;
    fn make_writer(&self) -> Self::Writer {
        MutexWriter(self.lock().unwrap())
    }
}

pub struct MutexWriter<'a, W>(MutexGuard<'a, W>);

impl<W: io::Write> io::Write for MutexWriter<'_, W> {
    // write to the shared writer in the `MutexGuard`...
}

Unfortunately, it's impossible to write this. Since MakeWriter always takes an &self parameter and returns Self::Writer, the generic parameter is unbounded:

    Checking tracing-subscriber v0.2.4 (/home/eliza/code/tracing/tracing-subscriber)
error[E0207]: the lifetime parameter `'a` is not constrained by the impl trait, self type, or predicates
  --> tracing-subscriber/src/fmt/writer.rs:61:6
   |
61 | impl<'a, W: io::Write> MakeWriter for Mutex<W>
   |      ^^ unconstrained lifetime parameter

error: aborting due to previous error

This essentially precludes any MakeWriter impl where the writer is borrowed from the type implementing MakeWriter. This is a significant blow to the usefulness of the trait. For example, it prevented the use of MakeWriter in tracing-flame as suggested in #631 (comment).

Proposal

This PR changes MakeWriter to be generic over a lifetime 'a:

pub trait MakeWriter<'a> {
    type Writer: io::Write;

fn make_writer(&'a self) -> Self::Writer;

The self parameter is now borrowed for the &'a lifetime, so it is okay to return a writer borrowed from self, such as in the Mutex case.

I've also added an impl of MakeWriter for Mutex<T> where T: Writer.

Unfortunately, this is a breaking change and will need to wait until we release tracing-subscriber 0.3.

Signed-off-by: Eliza Weisman

所以, 做这种破坏性变更的动机是, 实际实现这个trait的时候,遇到了一些麻烦, 比如当实现里面需要lifetime, 而 trait 定义本身并没有这个 lifetime 时, 编译通不地这.

Refs

https://docs.rs/tracing-subscriber/0.2.25/tracing_subscriber/fmt/trait.MakeWriter.html

https://docs.rs/tracing-subscriber/0.3.3/tracing_subscriber/fmt/trait.MakeWriter.html

https://github.com/LukeMathWalker/tracing-bunyan-formatter/commit/7acebe58a5ae3079e938a9a0a6dab1a7b5226692

https://github.com/LukeMathWalker/zero-to-production/blob/42d4f6a024fda2e7bc277679a595e3edfa2cb6c9/Cargo.toml#L28

Advanced Lifetimes https://doc.rust-lang.org/1.30.0/book/2018-edition/ch19-02-advanced-lifetimes.html#lifetime-bounds-on-references-to-generic-types

https://doc.rust-lang.org/rust-by-example/scope/lifetime/lifetime_bounds.html

https://doc.rust-lang.org/reference/trait-bounds.html

The “Advanced Lifetimes” section in Chapter 19 was removed because compiler improvements have made the constructs in that section even rarer. https://doc.rust-lang.org/beta/book/title-page.html

--EOF