A basic model that can be used on tabular data
Through trial and error, this general rule takes the lower of two values:
- A dimension space of 600
- A dimension space equal to 1.6 times the cardinality of the variable to 0.56.
This provides a good starter for a good embedding space for your variables. For more advanced users who wish to lean into this practice, you can tweak these values to your discretion. It is not uncommon for slight adjustments to this general formula to provide more success.
This model expects your cat
and cont
variables seperated. cat
is passed through an Embedding
layer and potential Dropout
, while cont
is passed though potential BatchNorm1d
. Afterwards both are concatenated and passed through a series of LinBnDrop
, before a final Linear
layer corresponding to the expected outputs.
emb_szs = [(4,2), (17,8)]
m = TabularModel(emb_szs, n_cont=2, out_sz=2, layers=[200,100]).eval()
x_cat = torch.tensor([[2,12]]).long()
x_cont = torch.tensor([[0.7633, -0.1887]]).float()
out = m(x_cat, x_cont)
Any direct setup of TabularModel
's internals should be passed through here:
config = tabular_config(embed_p=0.6, use_bn=False); config