flambe.nlp.language_modeling
¶
Submodules¶
Package Contents¶
-
class
flambe.nlp.language_modeling.
PTBDataset
(cache=False, transform: Dict[str, Union[Field, Dict]] = None)[source]¶ Bases:
flambe.dataset.TabularDataset
The official SST training dataset.
-
PTB_URL
= https://raw.githubusercontent.com/yoonkim/lstm-char-cnn/master/data/ptb/¶
-
classmethod
_load_file
(cls, path: str, sep: Optional[str] = 't', header: Optional[str] = None, columns: Optional[Union[List[str], List[int]]] = None, encoding: Optional[str] = 'utf-8')¶ Load data from the given path.
-
-
class
flambe.nlp.language_modeling.
LMField
(**kwargs)[source]¶ Bases:
flambe.field.TextField
Language Model field.
Generates the original tensor alongside its shifted version.
-
process
(self, example: str)¶ Process an example and create 2 Tensors.
Parameters: example (str) – The example to process, as a single string Returns: The processed example, tokenized and numericalized Return type: Tuple[torch.Tensor, ..]
-
-
class
flambe.nlp.language_modeling.
LanguageModel
(embedder: Embedder, output_layer: Module, dropout: float = 0, pad_index: int = 0, tie_weights: bool = False)[source]¶ Bases:
flambe.nn.Module
Implement an LanguageModel model for sequential classification.
This model can be used to language modeling, as well as other sequential classification tasks. The full sequence predictions are produced by the model, effectively making the number of examples the batch size multiplied by the sequence length.
-
forward
(self, data: Tensor, target: Optional[Tensor] = None)¶ Run a forward pass through the network.
Parameters: data (Tensor) – The input data Returns: The output predictions of shape seq_len x batch_size x n_out Return type: Union[Tensor, Tuple[Tensor, Tensor]]
-