A resident of South Orange, New Jersey, Gilman has attended Newark Academy. He is on his school's cross-country running and fencing teams and he enjoys golf. Gilman had his Bar Mitzvah in 2012. When eleven or twelve years old, he got the co-starring role of Sam Shakusky in Wes Anderson's 'Moonrise Kingdom', alongside Kara Hayward and an all-star cast of Bill Murray, Bruce Willis, Edward Norton, Harvey Keitel, Bob Balaban, Tilda Swinton and Frances McDormand.