Smart charging systems can reduce the stress on the power grid from electric vehicles by coordinating the charging process. To meet user requirements, such systems need input on charging demand, i.e., departure time and desired state of charge. Deriving these parameters through predictions based on past mobility patterns allows the inference of realistic values that offer flexibility by charging vehicles until they are actually needed for departure. While previous studies have addressed the task of charging demand predictions, there is a lack of work investigating the heterogeneity of user behavior, which affects prediction performance. In this work we predict the duration and energy of residential charging sessions using a dataset with 59, 520real-world measurements from 267 electric vehicles. While replicating the results put forth in related work, we additionally find substantial differences in prediction performance between individual vehicles. An in-depth analysis shows that vehicles that on average start charging later in the day can be predicted better than others. Furthermore, we demonstrate how knowledge that a vehicles charges over night significantly increases prediction performance, reducing the mean absolute percentage error of plugged-in duration predictions from over 200 % to 15%. Based on these insights, we propose that residential smart charging systems should focus on predictions of overnight charging to determine charging demand. These sessions are most relevant for smart charging as they offer most flexibility and need for coordinated charging and, as we show, they are also more predictable, increasing user acceptance.