(a) Regular convolution:
AlexNet/VGG
(b) Separable convolution block:
Split Regular convolution into Depth wise and Point wise.
(c) Separable with linear bottleneck:Import ResNet bottleneck into Separable convolution.
(d) bottleneck with expansion layer:
Invert bottleneck. (Small – Large – Small)