Hebbian cell assemblies can be formalised as sets of tightly connected cells in auto- and hetero-associative memories. Direct evidence for such “cliques” has recently been obtained in multiple-unit recordings from rat hippocampal neurons. These experiments suggest a hierarchical organisation where cliques are embedded in each other such that larger cliques represent less specific stimulus conditions. We here suggest an interpretation stating that the firing patterns may not just reflect nested categories but a lattice of concepts about stimulus–response mappings in the sense of formal concept analysis, an applied branch of set theory. We present an implementation of formal concept lattices in bidirectional associative memories that in contrast to previous work satisfies Dale’s principle and uses balanced excitation and inhibition. Inhibitory cells have fixed, non-plastic synapses even if the model learns new concepts. As an extreme case a single global inhibitory cell is enough that controls the total level of activation. The excitatory cells can further learn incrementally using a Hebbian coincidence learning rule. Implications of the model for retrieval in auto-associative memories are further outlined. Overall the model is well suited for representing hierarchical compositional relationships between entities in the form of correlated patterns in technical cognitive systems and potentially the brain.