Layer Keys are simply a way to "map" AEC Objects to predefined Layers with certain Layer Properties. I've just explained one reason why this is useful above. But there are other reasons too. First, it provides a great way to pre-define Company or Design Standards. All one has to do is place the Pipe or Duct or Conduit on the right "System" and all the proper Layer and Layer Properties are set. No more "What layer does CHWS Pipe go on? And what color is the Layer?" This is not tied to a DWG Template file or anything - you open any DWG file, initiate an MEP Object Tool and it falls on the correct Layer with the correct Layer Properties no matter what. Also, AEC Objects act differently than normal AutoCAD entities. You might have numerous different types or sized objects in a single command (placing a long run of duct, for instance). Well, AutoCAD MEP sees these AEC Obects, but knows it's tied to Systems, which changes the game. So say I place a long run of Supply Duct when it should have been Return Duct. I want to select a piece of Duct in the run, and change the System, not the Layer. The System is what drives the Layer, etc. If I were to simply change the Layer, the System stays the same and can really monkey things up.
What it truly boils down to is AutoCAD MEP is trying to recreate what Revit has created. But, AutoCAD at its core is married to Layers so this is why Layer Keys exist (my assumption anyways). Revit works the complete opposite. Duct is duct. Pipe is pipe. Conduit is conduit. Walls are walls. There's no Layers in Revit at all. Everything is categorized in a manner that relates exactly like the real world. AutoCAD can't recreate this completely so this is why I assume Layer Keys were put in place to merge the two types of approaches together.
Have you ever used the Tool Palettes for standard AutoCAD blocks and entities? If so, have you ever used the right-click > Properties option to preset the Layer, Linetype, Rotate on insert, etc? It's basically the same thing just a lot more advanced, and only with AEC Objects.
Hope this helps clear it up a bit.