Value alignment strategies in AI tend to assume that preferences are concrete manifestations of values, and preference-aligned machines are also value-aligned machines. However, scholars in philosophy increasingly note that holding this assumption may lead to shortcomings with alignment efforts. This brief will examine preferentism and human value theories as theoretical foundations for alignment, identify where the model breaks down and map new research directions in value alignment.
