Hierarchical Attention Networks in Text Categorizer

[alejandrojcastaneira]

Hello,

I understand that the text categorizer architecture it's similar to (Yang et al., 2016). I believe would be good to included an attention mechanism at sentences levels, for cases of long documents classification like your example on thinc imdb_cnn: https://github.com/explosion/thinc/blob/master/examples/imdb_cnn.py. I believe it will be very nice to have this, besides of the currently available: "ensemble", "simple_cnn" and "bow".

Best regards.

[honnibal]

I agree it could be good to have this. If you want to implement the model option, I would accept the PR.

[svlandeg]

As a quick note if anyone would start working on this: this should be developed for v.3.0 onwards (i.e. the current develop branch), see also #5143 for a recent PR related to the 3 existing textcat architectures.