I have the following pipeline:
sg = Pipeline([('tfidf', TfidfVectorizer()), ('normalize', Normalizer()), ('l1', LogisticRegression(penalty="l1", dual=False))])
and after peforming the fitting, I want to extract the tokens that correnponds to the non-zero weights.
How can I do this?
features = pipeline.named_steps['tfidf'].get_feature_names()
print(features[pipeline.named_steps['l1'].coef_ != 0])
See TfidfTransformer docs, LogisticRegression docs and the unmerged improved pipeline docs here