She came on the Ajira plane with Ben and Sun, Ilana and the rest but was flashed off the plane with Sayid, Hurley and Jack to the 1970s. That was NOT part of Smokey's plan. In the 70s, they set off the bomb that created the new timeline which, imo, was Jacob's "double-cross" in response to Smokey's machinations. Smokey doesn't know this has happened!
So maybe seeing Kate unsettles him because he doesn't know what her role is now. More proof, imo, that he doesn't know about the bomb being set off in the 70s and the new timeline coming into being. Jacob tricked him. That's why Jacob seemed resigned to his fate: he had found another "out". Also explains why Jacob wanted to get Hurley and Jack out of the temple.
Or not. LOL.
One problem with my theory: Sawyer and Miles were also stuck in the 70s and flashed out of there when the bomb went off. Does Smokey know how they got out of the 70s? He certainly hasn't seemed surprised at the presence of either Sawyer or Miles. Did he know Jack, Kate, Hurley and Sayid were on Ajira? Did he think they died? Did Jacob flash them to the 70s? So many questions...
What does it do to Smokey's plans for Sawyer that Kate now seems to be tagging along with his "followers"? I don't think she's a recruit yet.