Re: [asciidoc-lang-dev] Extension hooks in Asciidoc

[David Jencks <david.a.jencks@xxxxxxxxx>]

> On Mar 15, 2021, at 2:03 AM, Sylvain Leroux <sylvain@xxxxxxxxxxx> wrote:
> 
> On 13/03/2021 18:06, David Jencks wrote:
>>>> Maybe…. perhaps in the other direction I've considered trying to refactor Asciidoctor so that anything that looks syntax-wise like a extension is implemented as an extension.  I wonder if requiring this in the spec is reasonable.  Being able to replace behavior could be useful. There could be a set of required extensions.  As usual, needs more thought :-)
>>> 
>>> I have close to zero experience with extensions. Could you be more
>>> explicit regarding the standard features "that looks syntax-wise like an
>>> extension"?
>>> 
>> 
>> I'm thinking of things such as `image:…`, `xref:…` which look like inline macros and `image::…` that looks like a block macro.>
>> `include::…` needs to stay special as it does not necessarily introduce block content.
> 
> Thanks for the reply, David.
> 
> TL;DR:
> 
> Macro-like directives make good extension points. And not all processors
> might want to implement those features. So I support the idea of making
> them extensions.
> 
> 
> For the long message now:
> 
> This WE, I studied a little bit more AsciiDoc extensions. There appear
> ([1]) to have much more extension points than I believed it.
> 
> FWIW, the `image:…`, `xref:…` and `image::…` have in common to link to
> (or include from) external sources. I can imagine a situation where a
> client application might want to limit or filter what's allowed here.
> Say, for example, in the case of a Forum application using AsciiDoc as
> its message input format: the processor might want to enforce a forum's
> policy restricting the allowed image format or origin.

I don’t think it’s any business of a syntax extension, built-in or not, to interpret tokens referring to other resources. I think this functionality needs to be extracted to a separately replaceable document set service.  Refactoring Asciidoctor for a POC of this is proving to be... time consuming.

> 
> 
> On 13/03/2021 18:06, David Jencks wrote:
>> `include::…` needs to stay special as it does not necessarily introduce block content.
> 
> 
> As far as I understand, `include::...` are different because they are
> resolved before the document is parsed (in the pure tradition of the C
> pre-processor). That being said, a processor might want to limit or
> filter those directives at a finer granularity than currently allowed by
> the "safe-mode" security-level control [2].

I think all the “safe-mode” stuff is the responsibility of the document set. You are asking, ‘is this document in the document set?’.

The include:: preprocessor instruction might be more complicated than the C preprocessor, which I’m not really familiar with.  You can use attributes in the include:: instruction, which can be set externally, in the document header, in the document body before the include:: instruction, and by block and block macro processors ‘invoked’ before the include instruction.  I don’t recall verifying that attributes added by block/block macro processors are taken into account by include::, but I think they should be for logical consistency.
I think this means that the source has to be streamed through the parser perhaps line by line as each line may affect the content of the next line.  (lines may not be the correct unit here, perhaps blocks? include:: directives just need to start a line, but attributes can only be influenced at block boundaries). I think this also puts some constraints on how much lookahead is possible, since the entire “preprocessed source” isn’t known until parsing is finished.

I’ll mention that I think the current IncludeProcessor and Preprocessor extensions don’t really make sense, but I don’t have a clear idea how to deal with these areas in a better way.
> 
> Conditional directive [3] (`ifdef::..., endif::...`) should probably
> remain part of the core specs.
> 

Even I haven’t thought of a way to sensibly change or extend their functionality :-)
> 
> 
> On 13/03/2021 02:50, David Jencks wrote:
>> I wonder if requiring this in the spec is reasonable.  Being able to> replace behavior could be useful.  There could be a set of required
>> extensions.
> 
> Could there be a set of "standard extensions" whose behavior is defined
> in the specs but whose implementation is not mandatory?
> 

I’d need a concrete example to think this was a good idea.  There aren’t that many macro-like behaviors built in and pretty much all seem essential to me.

> 
> Regards,
> - Sylvain Leroux
> 
> 

Thanks!
David Jencks
> 
> [1]:
> https://www.rubydoc.info/github/asciidoctor/asciidoctor/Asciidoctor/Extensions
> [2]: https://docs.asciidoctor.org/asciidoctor/latest/safe-modes/
> [2]: https://docs.asciidoctor.org/asciidoc/latest/directives/conditionals/
> _______________________________________________
> asciidoc-lang-dev mailing list
> asciidoc-lang-dev@xxxxxxxxxxx
> To change your delivery options, retrieve your password, or unsubscribe from this list, visit
> https://dev.eclipse.org/mailman/listinfo/asciidoc-lang-dev